I think i have got the answer from xnio source code.

    synchronized(this.workLock) {
        this.selectorWorkQueue.add(command);
        Log.log.tracef("Added task %s", command);
    }

the problem will occur like these steps:

first, virutal thread A call the log method of logback(which use ReentrantLock), wait for io finish, it umounted from the os thread(from ForkJoinPool).

other virtual thread call the code above to close a socket, the key word "synchronized" caused ForkJoinPool thread been held by virtual thread.

virutal thread A finished it's io work and want to realese the log lock, but there is not available os thread in ForkJoinPool to execute.

why ForkJoinPool did not expand it's pool size? i guess the accept thread was waiting for log lock, so there is no more virutal thread been submitted