Yes, you are right. Traditionally, garbage collection (GC) in Java (and the CLR) could cause a "stop-the-world" pause. When this GC runs, you can observe that all application threads are paused. This happens particularly during heap compaction (i.e., when live objects are moved around in memory, and references need to be updated safely to point to the new memory location). When the threads are suspended, it prevents issues like accessing an object in the middle of being moved.
However, now the modern JVMs (like HotSpot) have introduced two types of GCs: concurrent and parallel garbage collectors. These GCs actually minimize "stop-the-world" pause. For example,
G1 GC and ZGC are designed to perform most GC work concurrently, (i.e.) the application threads never stop and keep running even when the GC is happening in the background.
These GCs split the heap into regions, and only a few regions are compacted at a time.
Safepoints (short pauses) still occur, but they're significantly reduced in duration.
This could be the reason how enterprise grade apps like JBoss or GlassFish maintain high throughput and responsiveness. They actually rely on these low-pause collectors and tune JVM GC settings for production environments.
In case you are interested, this blog post on GC fundamentals discusses how garbage collection works under the hood, and it explains the evolution from traditional to modern GCs like G1 and ZGC. It is a useful resource that provides a great overview and deeper insight.
As for NUMA (Non-Uniform Memory Access), currently JVMs are increasingly becoming NUMA-aware, and they optimize memory allocation to reduce latency when accessing memory local to a thread’s CPU.