The proposed setup in my question works perfectly. I realized I hadn't restarted the Caddy container for a while. When I checked the Caddyfile, it actually contained some lines from a previous attempt at getting fonts working:

@fonts { 
path .woff *.woff2 *.ttf *.eot *.svg path_regexp \.(woff|woff2|ttf|eot|svg)$ 
}

handle @fonts { 
header Cache-Control "public, max-age=31536000" 
header Access-Control-Allow-Origin "" 
file_server 
} 

Removing this and restarting the Caddy container with the Caddyfile I provided in the question worked.