The 80 column IBM punch cards can contain binary data with each column representing a full binary byte. I used this all the time to output binary data. A compiler or assembler can produce and Object deck of cards, which is fed into a linker for execution. The obj was binary information. The object deck had different record types, the most typical format was the TXT record (https://www.ibm.com/docs/en/hla-and-tf/1.6?topic=output-txt-record-format)

there also was a REP type which allowed you to patch an object deck on the fly. I did this on an IBM 360 & 370 machines