Just in case somebody has the same issue.
It was actually caused by the server process model of FastAPI (Multiple Uvicorn Workers) in my case.
The implementation suggested here, solved my issue: https://prometheus.github.io/client_python/multiprocess/