I am not a expert in PINN. But I think the problem is about extrapolation. Like a polynomial a neural network is NOT a genetic function approximator. You may use a Fourier Layer. This can help.