Turns out you need a docker config json secret to make it work.

Which can be created like so:

kubectl create secret generic registry-gitlab-com-repo-auth \
 --namespace crossplane-system \
 --from-file=.dockerconfigjson=config.json \
 --type=kubernetes.io/dockerconfigjson

The config.json file (refered to in the above statement) should look like this:

{                                                                                                                                                                                                                                                                                                                                                                        
    "auths": {                                                                                                                                                                                                                                                                                                                                                                              
        "registry.gitlab.com": {                                                                                                                                                                                                                                                                                                                                                            
            "auth": "<base64_encoded_username_password_combination_like_in_basic_auth>"                                                                                                                                                                                                                                                                                      
        }                                                                                                                                                                                                                                                                                                                                                                                   
    }                                                                                                                                                                                                                                                                                                                                                                                       
}

To create the base64 encoded auth string on the command you could use:

USER=foo
PASS=bar
echo -n $USER:$PASS | base64  # should output the base64 encoded string