I used wireshark to sniff the communication - ModemManager overwrites APN setup each time.
ModemManager
Solution was to use qmicli, which creates profiles, and you can just choose which one to connect.
qmicli