Thank you all for your suggestions!
I finally solved the problem by removing the resize mask and implementing the zoom functionality myself.
I had actually tried this approach before, but initially encountered issues with the zoom functionality becoming unavailable. This led me to explore alternative solutions, such as handling WM_HITTEST and returning HT_CLIENT similar to how it's done in Windows. However, I couldn't find a viable solution using this method.