Just to add to the answer, resizing your image does help, the issue you have to have in mind is how much memory OpenCV is consuming.

In my case, scaling down the images helped up until a certain point, we are doing multiple threads to run concurrent SIFT feature matching logic and we would still have the restart happening depending on how many concurrent requests we had. So take a look at the machine overall memory as well and find the sweet spot. In our case, we were able to reduce JVM max memory to allow more memory for OpenCV to do its job and were able to have more concurrent threads running.

Don't forget to release your Mats as well with mat.release()