I found the solution, I changed to UnsqueezeTransform(-4, in_keys=["pixels"]) within agent_explore and now I have the wanted behaviour ... (:
UnsqueezeTransform(-4, in_keys=["pixels"])