Posting my solution in case it helps anyone else. I needed to add the bearer token as a authentication scheme like this:

builder.Services.AddAuthentication()
    .AddBearerToken(IdentityConstants.BearerScheme);

builder.Services.AddIdentity<ApplicationUser, IdentityRole>()
    .AddEntityFrameworkStores<ApplicationDbContext>()
    .AddApiEndpoints()
    .AddDefaultUI()
    .AddDefaultTokenProviders();

I then needed to allow anonymous access to those API endpoints:

app.RegisterApiEndpoints();
app.MapIdentityApi<ApplicationUser>().AllowAnonymous();

Finally I needed to add a custom policy that accepted the bearer token and the cookie to the endpoints that were going to be accessed with the bearer token. The answer here really helped with that: Why am I redirected to Account/Login page when using WebApi and AspIdentityCore beside JwtBearer in dotnet 7?