With help from @NKK who identified the issue in the comments, I managed to resolve this issue, and it was related to the HttpMessageHandler lifetime in the HttpClient configuration.
The problem occurred because the HttpMessageHandler was being disposed after its default lifetime of about 2 minutes. Since I was manually storing cookies in a dictionary within this handler, when the handler was disposed and recreated, my custom cookie storage was lost, causing the Set-Cooki headers to disappear.
The fix was to explicitly set a longer lifetime for the HttpMessageHandler in the dependency injection configuration:
services.AddHttpClient<IClient, Client>(c =>
{
c.BaseAddress = new Uri(configuration.GetSection("ApiAddress").Value!);
})
.AddHttpMessageHandler<AuthenticatedHttpClientHandler>()
.SetHandlerLifetime(TimeSpan.FromHours(6))
The .SetHandlerLifetime() method prevents the handler from being disposed and recreated approximately every 2 minutes, which ensures that my manually maintained cookie dictionary persists for the specified duration.