Reports

The Root Cause: File Descriptor Exhaustion

The 502 Bad Gateway errors you're experiencing with your FastAPI application under load are most likely caused by file descriptor exhaustion. This is a common issue when running Uvicorn (or other ASGI servers) behind a reverse proxy like Nginx.

I've created a complete proof-of-concept that demonstrates this issue in great detail and confirms that file descriptor exhaustion directly causes 502 errors.

What are File Descriptors?

File descriptors (FDs) are numeric identifiers for open files, sockets, and other I/O resources. Each connection to your application uses at least one file descriptor, and there's a limit to how many a process can have open simultaneously.

When your application runs out of available file descriptors:

It can't accept new connections
It may fail to open new files or sockets
The reverse proxy (Nginx) can't establish a connection to your application
Nginx returns a 502 Bad Gateway error to the client

How I Verified This Is the Cause

I created a test environment with:

A FastAPI application that can intentionally leak file descriptors
Nginx as a reverse proxy
A test script that incrementally leaks FDs while making requests

The results clearly show that once file descriptor usage approaches the limit, Nginx starts returning 502 Bad Gateway errors.

Here's the relevant output from my test:

[  1] ✅ OK (0.01s) - FDs: 12/50 (24%), Leaked: 3
[  2] ✅ OK (0.01s) - FDs: 16/50 (32%), Leaked: 6
...
[ 13] ✅ OK (0.01s) - FDs: 49/50 (98%), Leaked: 39
[ 14] ✅ OK (0.01s) - App error: HTTPConnectionPool(host='localhost', por...
[ 15] ⛔ 502 BAD GATEWAY (0.12s) - FDs: 49/50 (98%), Leaked: 41
...

As you can see, once file descriptors approach 100% of the limit, 502 errors start occurring.

Common Scenarios That Lead to File Descriptor Exhaustion

Resource leaks: Not properly closing files, connections, or sockets
High concurrent load: Too many simultaneous connections
Low system limits: Default file descriptor limits are too low
Long-lived connections: WebSockets or other long-running connections
Database connection pools: Improperly configured pools that open too many connections

How to Fix the Issue

1. Increase File Descriptor Limits

In production environments, increase the file descriptor limits:

For systemd services:

# /etc/systemd/system/your-service.service
[Service]
LimitNOFILE=65535

For Docker containers:

# docker-compose.yml
services:
  app:
    ulimits:
      nofile:
        soft: 65535
        hard: 65535

For Linux systems:

# /etc/security/limits.conf
your_user soft nofile 65535
your_user hard nofile 65535

2. Implement Protective Middleware

Add middleware to monitor file descriptor usage and return controlled responses when approaching limits:

import resource
from fastapi import Request, Response
from starlette.middleware.base import BaseHTTPMiddleware

class ResourceMonitorMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request: Request, call_next):
        # Get current FD count and limits
        soft_limit, _ = resource.getrlimit(resource.RLIMIT_NOFILE)
        fd_count = len(os.listdir('/proc/self/fd')) - 1  # Subtract 1 for the listing itself
        
        # If approaching limit, return 503
        if fd_count > soft_limit * 0.95:
            return Response(
                content="Service temporarily unavailable due to high load",
                status_code=503
            )
        
        # Otherwise process normally
        return await call_next(request)

# Add to your FastAPI app
app.add_middleware(ResourceMonitorMiddleware)

3. Fix Resource Leaks

Make sure you're properly closing all resources:

# Bad - resource leak
def bad_function():
    f = open("file.txt", "r")
    data = f.read()
    return data  # File is never closed!

# Good - using context manager
def good_function():
    with open("file.txt", "r") as f:
        data = f.read()
    return data  # File is automatically closed

4. Configure Connection Pooling

Properly configure connection pools for databases and external services:

from sqlalchemy import create_engine
from databases import Database

# Configure pool size appropriately
DATABASE_URL = "postgresql://user:password@localhost/dbname"
engine = create_engine(DATABASE_URL, pool_size=5, max_overflow=10)
database = Database(DATABASE_URL)

5. Set Appropriate Timeouts

Configure timeouts in both Uvicorn and Nginx:

Uvicorn:

uvicorn app:app --timeout-keep-alive 5

Nginx:

http {
    # Lower the keepalive timeout
    keepalive_timeout 65;
    
    # Set shorter timeouts for the upstream
    upstream app_server {
        server app:8000;
        keepalive 20;
    }
    
    location / {
        proxy_connect_timeout 5s;
        proxy_read_timeout 10s;
        proxy_send_timeout 10s;
    }
}

How to Monitor File Descriptor Usage

In Production

Add monitoring for file descriptor usage:

import psutil
import logging

def log_fd_usage():
    process = psutil.Process()
    fd_count = process.num_fds()
    limits = resource.getrlimit(resource.RLIMIT_NOFILE)
    
    logging.info(f"FD usage: {fd_count}/{limits[0]} ({fd_count/limits[0]:.1%})")
    
    if fd_count > limits[0] * 0.8:
        logging.warning("High file descriptor usage detected!")

For Debugging

To check file descriptor usage:

# For a specific PID
lsof -p <pid> | wc -l

# Check limits
ulimit -n

Conclusion

502 Bad Gateway errors in FastAPI/Uvicorn applications are commonly caused by file descriptor exhaustion. By monitoring FD usage, increasing system limits, and implementing protective middleware, you can prevent these errors and maintain a stable application even under high load.

The key to resolving this issue is proper resource management and monitoring, ensuring that your application can gracefully handle load without exhausting system resources.

Code for the complete proof-of-concept is available in this repository, including a FastAPI application, Nginx configuration, and test scripts to demonstrate and resolve the issue.

79516441