Answering my own question, so others having same issue might find a easier solution than spending 3 days debugging.
Apparently this is intended as of sometime in 2017 where chromium introduced this as a feature to save resources ( wich is a valid point ) - see here: https://issues.chromium.org/issues/40492976 Sadly this breaks user expectations, if i have a videoelement with autoplay = true, it sends an pause event if the videoelement is not in view (either detached, another tab, or if its just out of scroll view). This should be expected to play the video track instead of pausing it behind the scenes.
Another resource about this implementation is here: https://developer.chrome.com/blog/media-updates-in-chrome-61#background-video-track-optimizations
Initially i tried canceling the onpause event but that didnt work so i tried the following that both worked:
1) Append it to the document body, layer it on top of my canvas and set visibility:hidden; (display:none fails with same reasoning, so use visibility:hidden)
2) when stream is optained, and the detached videoelement is created, set the src = stream & call videoElement.play() manually
I ended up using uption 2
This felt like the most sane workaround, i dont manipulate any pause events that might be used elsewhere, and i dont have to append the videoelement and hacky position it over my canvas to keep it shown in view with the canvas at all times while still hiding it. here i simply play manually, wich doenst work automatically with autoplay in this situation where pause events are sent.
it still feels hacky, and weird that firefox handles this in a much better way, atleast autoplay = true, should not care about the videoelement being detached or not.