Possibly xorriso -as mkisofs option -boot-info-table is to blame. It causes xorriso to overwrite bytes 8 to 63 in the boot image. Larger boot images use these bytes to locate themselves on the optical boot medium and to load more of themselves than just the 4 blocks of 512 bytes, which they trust any BIOS to be able to load.
So try xorriso -as mkisofs without option -boot-info-table. Better omit option -boot-load-size 4 , too, until your bootimage becomes too large for getting loaded by your BIOS. (Then you will have to study ISOLINUX or GRUB for their preparation and use of bytes 8 to 63 in the boot image.)