I had the same problem and made ticket in the zmk github project. https://github.com/zmkfirmware/zmk/issues/2573
You'll want to specify a chosen matrix transform, e.g. add zmk,matrix-transform = &default_transform; --> See ticket for more details and why you need to do so.