As others mentioned, it will be tough to understand the problem without heap dump. You can verify below and if any of them is true that may be one of the causes to the issue.

1. Is there any objects being initialised during application startup. If yes check the size of the objects being loaded and avoid loding high payload into memory.
2. Are you loading any objects in memory and adding objects to the same objects which kept in memory.
3. There is possibility that any third party library could have stored high payload in memory. If that's the case then you might not be using it correctly.

However, I would still recommend to capture the heap dump and check the area which is consuming high memory and handle.

If it is feasible to connect to visualvm connect to it and there you can analyse or capture heap dump and analyse it using Eclipse-MAT app.

May be this link helps in capturing heap dump: How to take heap dump?

Hope this helps to understand the problem.