I was struggling with this issue for a few days. Not sure if my Firebase setting was wrong or if my code was wrong.

The it was resolved by the following steps.

1. In the Credentials -> OAuth 2.0 Client IDs, find the Web Application you are using, get in and add https://yourcustomdomain/__/auth/handler in the Authorized redirect URIs.

2. In your firebase auth Config, change authDomain to your custom domain.

3. *** Add the following to the nginx.

   location /__/auth { proxy_pass https://.firebaseapp.com; }

Without the step 3, you will get a 404 on https://yourcustomdomain/__/auth/handler.

For the solution, you may refer to the following web site. I am not sure if my problem was caused by the browsers that block third-party cookies. But the solution saved my life!

https://firebase.google.com/docs/auth/web/redirect-best-practices?_gl=1*kqfq05*_up*MQ..*_ga*NzgxNDg1MzYxLjE3Mzc1OTIzNDQ.*_ga_CW55HF8NVT*MTczNzU5MjM0NC4xLjAuMTczNzU5MjM0NC4wLjAuMA..