You receive the error [mpegts @ 0x7fc4c00343c0] AAC bitstream not in ADTS format and extradata missing probably because you haven't called avcodec_parameters_from_context(), which copies the AAC codec extradata from the codec context to AVStream codecpar. If it's there, the mpegts muxer will create another AVFormatContext for this purpose by itself.