I think the issue is that Uvicorn (if you're using it) does not trust proxy headers by default unless specified.
According to the FastAPI documentation about deploying FastAPI on Docker behind a proxy, you need to enable proxy headers.
Additionally, Starlette provides information about Uvicorn middleware for handling proxy header. The Uvicorn GitHub code also provides insight into how proxy headers are processed.