The solutions ended up being that I could pass an object of ThreadPoolTaskExecuter to the JmsListenerContainerFactory with the following parameter settings:
executor.setCorePoolSize(1);
executor.setMaxPoolSize(1);
executor.setQueueCapacity(100);
executor.initialize();
This has the desired behaviour.