I managed to crack this over the weekend. Turns out Application Insights had injected an environment variable for the java agent. I deleted the environment variable from the web app entirely and explicitly set the value of JAVA_TOOL_OPTIONS to "" in my docker file and the application booted!