You could debug the problem by

1. opening the Chrome Devtools > Network tab,

2. picking any Guacamole WebSocket connection

3. and inspecting the Timing sub-tab. It will look like the one on the screenshot. Here you can see that queueing happened to be very long (7.38 seconds!). Why? Because I had 5+ concurrent TCP connections (which is used by WebSockets) to the same resource.

From the docs:

There are already six TCP connections open for this origin, which is the limit. (Applies to HTTP/1.0 and HTTP/1.1 only.)

p.s. It is no solution nor advice but it might help you with the debugging (locating the problem). Or, at least, I hope it can solidify your understanding about the issue.