After being provided some help with the Azure support team, it appears that the error came from DPS not being able to communicate with the linked IoT Hub. That link was broken, but no flag is given on the DPS so I had no chance to find the issue.

The user friendly way of recovering this is to delete the link from the DPS tab, and recreate it. With Azure CLI, here's how to update the iot hub connection string registered by DPS:

az iot dps update --name MyExampleDps --set properties.iotHubs[0].connectionString="HostName=MyExampleHub-2.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=NewTokenValue"

(Taken from documentation https://learn.microsoft.com/en-us/azure/iot-dps/how-to-manage-linked-iot-hubs?tabs=cli#update-keys-for-linked-iot-hubs )