79662633

Date: 2025-06-11 20:41:09
Score: 3
Natty:
Report link

Resolving Window Message Blocking During Window Drag and Resize Operations in Windows

Cause and Effect

When you start dragging a window’s title bar to move it or adjust its size by dragging its border, your program first receives a WM_ENTERSIZEMOVE message. This message simply notifies you that dragging is about to begin. Subsequently, a WM_SYSCOMMAND message arrives, where the wParam value determines the action type. You’ll use switch (GET_SC_WPARAM(wParam)) to handle it. When the switch hits case SC_MOVE: or case SC_SIZE:, it indicates that the system is entering drag mode. If you directly forward this message to DefWindowProc(), it blocks by starting its own internal message loop, handling mouse movements, window resizing, and other complex operations internally. When you release the mouse, DefWindowProc() returns 0, ending the blocking. Finally, it sends a WM_EXITSIZEMOVE message to notify you that drag mode has ended.

For scenarios like game development, video player implementations, etc.—where you cannot rely on blocking-style GetMessage() loops with timers (Timer callbacks) and must avoid blocking during message processing (e.g., DefWindowProc() blocking)—yet still need to support window dragging and resizing, here’s a workaround technique to interrupt and recover from blocking.

Solution Approach

Interface Design

First, my window message handler is non-blocking. It’s named PollWindowEvents() and is called repeatedly by the user’s main loop. The caller handles its own tasks (e.g., rendering dynamic content) and then invokes PollWindowEvents() to process window messages, check keyboard inputs, etc. Similar to GLFW’s design, as long as PollWindowEvents() doesn’t block, the caller’s loop keeps running.

The logic of PollWindowEvents() roughly looks like this:

while (PeekMessageW(&msg, NULL, 0, 0, PM_REMOVE))
{
    TranslateMessage(&msg);
    if (msg.message == WM_QUIT)
    {
        w->should_quit = 1;
        w->exit_code = (int)msg.wParam;
    }
    else
    {
        DispatchMessageW(&msg);
    }
}

Key Insight

During window dragging, although DefWindowProc() blocks, timer events (WM_TIMER) still reach your window procedure. Additionally, WM_MOVE, WM_SIZE, and WM_SIZING messages are delivered frequently during drag/resize operations.

If within these messages, I can break out of DefWindowProc()’s context and make PollWindowEvents() return, blocking is resolved. On the next call to PollWindowEvents(), I jump back into the message handler as if nothing happened, resuming DefWindowProc()’s drag logic to complete the cycle.

Technical Implementation

Achieving this jump requires setjmp() and longjmp() from the C standard library. Unfortunately, they’re problematic here. MSVC’s longjmp() enforces RAII resource cleanup rules in C++ and handles uncaught exceptions. It assumes longjmp() is a non-returning operation, requiring all stack resources and exceptions to be resolved before jumping. This "safety" conflicts with our need, where jumping back into the context later shouldn’t disrupt normal RAII or exception handling.

Jumping is only part of the problem. The blocking DefWindowProc() uses its stack frame, while returning from PollWindowEvents() lets the caller use its stack frame. If we longjmp() out (which restores the stack pointer to the setjmp() state), two stack frames are used simultaneously—leading to data corruption and stack overwrites. Thus, we need two separate stacks.

Implementation Framework

  1. Replace longjmp(): Bypass MSVC’s extra safety checks using custom shellcode.
  2. Allocate a new stack: Use _aligned_malloc() to allocate a 16-byte aligned memory block as the new stack.
  3. Create a trampoline function, jmp_to_new_stack(), which does three things:
    • Switches the stack pointer to the top of the new stack.
    • Invokes a callback function (executed on the new stack).
    • Uses a custom my_longjmp() to return from the new stack to the old stack’s setjmp() position.

Why my_longjmp() instead of restoring the old stack pointer directly? Because both stacks are active; the original stack pointer becomes invalid after leaving its context.

  1. Setting jump targets:

    • At the start of PollWindowEvents() (before PeekMessageW), use setjmp() to set a "re-entry point" for resuming blocked operations.
    • At the end of PollWindowEvents() (after the PeekMessageW loop), use setjmp() to set an "escape point" for breaking out of blocking.
    • Inside WM_TIMER/WM_MOVE/WM_SIZE handlers, use setjmp() to set "re-entry points" for jumping back into the blocking function.

  2. Entering blocking mode:

    • In the WM_SYSCOMMAND handler (for SC_MOVE/SC_SIZE), create a timer, allocate the new stack, then call jmp_to_new_stack(). The provided callback invokes:
    DefWindowProcW(w->Window, WM_SYSCOMMAND, wParam, 0); // Enters blocking

  3. Escaping blocking mode:

    • Inside WM_TIMER/WM_MOVE/WM_SIZE handlers (executing on the new stack), use my_longjmp() to jump to the "escape point" at PollWindowEvents()’s end—returning control to the caller.

  4. Re-entry logic:

    • After escaping mid-message (without RAII/exceptions), the next PollWindowEvents() call detects the interrupted state and uses my_longjmp() to jump back to the handler’s "re-entry point."

  5. Exiting blocking mode:

    • When DefWindowProcW() returns (after drag ends), use my_longjmp() to jump to PollWindowEvents()’s start, letting normal message processing resume.

Code Implementation

Core Functions

The arrays contain shellcode (machine code) for context switching. Implementations:

Key Shellcode Snippets

Integration

  1. State Tracking Structure:

    typedef struct
{
    void *new_stack;                        // Allocated stack memory
    size_t new_stack_size;                  // Stack size
    volatile int hack_is_on;                // 1 if in blocking escape mode
    volatile int is_returned_from_timer;    // Flag for escaped state
    WPARAM syscommand_wparam;               // Saved WM_SYSCOMMAND param
    jmp_buf jb_returning;                   // Escape point (Poll end)
    jmp_buf jb_reentering;                  // Re-entry point (handlers)
    jmp_buf jb_exit_hacking;                // Exit point (blocking ends)
}HackWayAntiBlocking;

  2. Code to be added into PollWindowEvents() before PeekMessage():

    if (setjmp(w->hack.jb_exit_hacking) == 1)
{ // After blocking ends (DefWindowProc returns)
    KillTimer(w->Window, 1);
    w->hack.hack_is_on = 0;
}
if (w->hack.hack_is_on)
{ // Re-enter blocked context if needed
    if (w->hack.is_returned_from_timer)
        _my_longjmp(w->hack.jb_reentering, 1);
}

  3. Code to be added into PollWindowEvents() before returning:

    if (setjmp(w->hack.jb_returning) == 1)
{ // Escape from the blocked call
    return;
}

  4. WndProc Handler Additions:

    • On WM_SYSCOMMAND:
    case WM_SYSCOMMAND:
switch (GET_SC_WPARAM(wp))
{
case SC_MOVE:
case SC_SIZE:
    w = (void *)GetWindowLongPtrW(hWnd, 0);
    assert(w->hack_is_on == 0);
    w->hack.syscommand_wparam = wp;
    if (!w->hack.new_stack)
    { // Allocate memory for the new stack, 64 KiB is not too much or too less
        w->hack.new_stack_size = (size_t)1 << 16;
        w->hack.new_stack = _aligned_malloc(w->hack.new_stack_size, 16);
    }
    if (w->hack.new_stack)
    { // From here, we are going to switch to the new stack and call the blocking function.
        w->hack.hack_is_on = 1;
        // Start timer. When blocking, our `WndProc()` still can receive `WM_TIMER` event.
        SetTimer(w->Window, 1, 1, NULL);
        _jmp_to_new_stack(w->hack.new_stack, w->hack.new_stack_size, _run_blocking_proc, w, w->hack.jb_exit_hacking, 1);
    }
    else
    { // Can't allocate memory for the new stack? Give up here.
        w->hack.new_stack_size = 0;
    }
    break;
default:
    return DefWindowProcW(hWnd, msg, wp, lp);
}
break;
    • On WM_TIMER/WM_MOVE/WM_SIZE:
    case WM_MOVE:
case WM_SIZING:
case WM_SIZE:
case WM_TIMER:
    if (msg != WM_TIMER || wp == 1)
    {
        w = (void *)GetWindowLongPtrW(hWnd, 0);
        if (w->hack.hack_is_on)
        {
            int j = setjmp(w->hack.jb_reentering); // Set re-entry point and escape blocking
            switch (j)
            {
            case 0:
                // Escape blocking
                w->hack.is_returned_from_timer = 1;
                _my_longjmp(w->hack.jb_returning, 1);
                break;
            case 1:
                // Re-entry to here
                break;
            default:
                assert(0);
            }
            w->hack.is_returned_from_timer = 0;
        }
    }
    break;
    • The _run_blocking_proc():
    // Callback running on the new stack
void _run_blocking_proc(WindowsDemoGuts *w)
{ // Blocks here
    DefWindowProcW(w->Window, WM_SYSCOMMAND, w->hack.syscommand_wparam, 0);
}

  5. Cleanup:

    _aligned_free(w->hack.new_stack);
w->hack.new_stack = 0;
w->hack.new_stack_size = 0;

With these modifications, dragging/resizing a window won’t block PollWindowEvents()—it returns immediately—allowing the caller’s loop to maintain smooth rendering and audio playback.

Full Implementation

Complete source code:

Reasons:
  • RegEx Blacklisted phrase (2.5): Please provide your
  • Contains signature (1):
  • Long answer (-1):
  • Has code block (-0.5):
  • Contains question mark (0.5):
  • Low reputation (0.5):
Posted by: 0xAA55