Turns out replacing ConfigureFunctionsWorkerDefaults() with ConfigureFunctionsWebApplication() fixes my problem. I do not know what these function does and why the first gives broken http responses. Would love if someone could explain a bit!
ConfigureFunctionsWorkerDefaults()
ConfigureFunctionsWebApplication()