I was able to solve the issue by myself, but posting this here in case someone is having the same issues. I have added this in the Dockerfile for the build and publish stages. This can be used in the Jenkins pipeline as well as it throw the same error.

/p:UseAppHost=false

It turned out that this can be also places in the csproj file as well as I did yesterday:

<UseAppHost>false</UseAppHost>

However the wwwroot folder was still missing, so I have added a line in the Dockerfile for that as well:

COPY --from=publish /app/publish/wwwroot ./wwwroot

Where /app/publish is the same path set in the publish stage.

After these changes, everything is working as expected.