I finally got it working. For people that experience the same issues here are my detailed settings and requests.:

Public Client: app
Confidential Client: bridge

Grant permissions according to this manual: https://www.keycloak.org/securing-apps/token-exchange#_internal-token-to-internal-token-exchange

For public to public exchange:

curl -X POST \
--location 'https://mykeycloakurl.com/realms/myrealm/protocol/openid-connect/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'client_id=app' \
--data-urlencode 'grant_type=urn:ietf:params:oauth:grant-type:token-exchange' \
--data-urlencode 'requested_token_type=urn:ietf:params:oauth:token-type:refresh_token' \
--data-urlencode 'scope=<scope to be included>' \
--data-urlencode 'subject_token=<user app token>'

For public to confidential (should be handled by a backend system to protect the client secret):

curl -X POST \ 
--location 'https://mykeycloakurl.com/realms/myrealm/protocol/openid-connect/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'client_id=bridge' \
--data-urlencode 'client_secret=ytN5ZmXorCo3772yXAVXNIbualdYtvtm' \
--data-urlencode 'grant_type=urn:ietf:params:oauth:grant-type:token-exchange' \
--data-urlencode 'requested_token_type=urn:ietf:params:oauth:token-type:refresh_token' \
--data-urlencode 'subject_token=<user app token>'

Important the requests are POST.
And if you want to downgrade the scopes like @Gary Archer said you also need to manage the Scope policies accordingly. You can also specify a scope when exchanging public to confidential.