The problem in my code is that I was running the task twice.

//Process.Start will start the process.
var process = Process.Start(new ProcessStartInfo
    {
        FileName = "dotnet",
        Arguments = $"clean \"{csprojPath}\" -c Release",
        RedirectStandardOutput = true,
        RedirectStandardError = true,
        UseShellExecute = false,
    });
    
//2nd time here.
    process.Start();

This was causing the file access error. commenting the process.Start fixes the problem.