Finally I was able to solve the issue. The issue was mainly coming because the built-in agent was used. After that I have created a new Agent and when I looked into it - there were a couple of commands to run. After that the Agent is coming online and when he is running the build - I didn't experience the issue. However the problem is that whenever you end the session the agent goes offline again. I have followed this tutorial:

[How to setup agent in Jenkins][1]

This has fixed all of my issues with the Agent and building and testing the .net application. [1]: https://www.jenkins.io/blog/2022/12/27/run-jenkins-agent-as-a-service/