Mike's answer was the one I needed, however I was getting this error:

ArgumentNullException: IDW10106: The 'ClientId' option must be provided.

So I needed to manually bind the AzureAd config section to the identityOptions:

.AddMicrosoftIdentityWebApp(identityOptions => 
      /* {identityOptions.ClientId ="";}, // if you want to specify the options manually instead of Configuration.GetSection() call*/
      builder.Configuration.GetSection("AzureAd").Bind(identityOptions),