Found the solution. Indeed if you have a look at the question and why I needed ffmpeg for the "external" videos to play, you can figure out that there might be an audio problem somewhere. Well, even though I don't play audio on the device I run my application, the video still has audio.

Now going further with the investigation, I found that Rocky Linux uses PipeWire as its audio driver (together with Alsa, but that didn't matter) and found the environment variables that PipeWire has for my user. Turns out it was XDG_RUNTIME_DIR - from here. Got the variable that my user had with the set command and now it's all up and running.

Thank you for the suggestions, @Alex Crichton and @user1934428. You kinda pointed me in the right direction, because at first I didn't think it could be an env var missing.