Found a solution thanks to the comment from jonrsharpe:

By adding a tab character directly before the +kubebuilder part, it is possible to prevent gofmt from replacing the quotes.

// +kubebuilder:...

=>

//  +kubebuilder:...