In my case, the problem was that Linux was trying to optimize file writing to the disk by writing them to RAM (so the RAM was used up not by my Java application). Therefore, I rewrote the solution so that it processes the file in a streaming manner instead of writing it to disk, and that helped.