I finally figured out the approach.

1. So first you have to make a SUBDOMAIN for your website and add it to Firebase hosting. It will show you the steps to add it to your domain DNS records. You can have it like auth.domain.com
   You don't need to host your site there. Mine was being hosted on some other platform too. It's just to host the auth screen I guess.

2. Then you have to go to google cloud console -> google auth platform -> client ID

   

3. Then in the authorised JavaScript origins enter your domain and subdomain

4. In the Authorised redirect URIs enter auth.domain.com/__/auth/handler with HTTPS
   Here auth is the subdomain that I chose, you can choose any.

5. Then in your website in the env variable for FIREBASE_AUTH_DOMAIN enter auth.domain.com

6. Add your website info in Branding section. Thats what will appear on the login screen.

Idk why all this is not mentioned in the docs anywhere. Maybe its mentioned but very hard to find. Hopefully it helps someone. Note: I am using sign in with popup. Not sure if it works the same way with redirect.