The pyav module, based on ffmpeg, is capable of recording videos with an alpha channel, provided the right codecs are specified. The 'moviepy' and 'cutcutcodec' libraries offer a higher-level interface for this purpose.

There is an example here: https://cutcutcodec.readthedocs.io/latest/build/examples/advanced/write_alpha.html