I've met a similar issue while putting a Button next to a Picker in the same HStack which is contained in a Form.

This Apple developer forum thread brings me the solution/workaround: simply add .buttonStyle(BorderlessButtonStyle())