Removing passenger_enabled on; from the server block and adding:

location / {
    try_files $uri.html $uri @passenger;
}
location @passenger {
    passenger_enabled on;
}

Thanks to Camden Narzt and Frank Groeneveld for pointing me in the right direction