do I need to have the authentication provider configured in order to use the identity db?

Yes, you need it, but if just for set the authentication provider, it's easy to do it. Just install the related package and add the service.

If you have trouble for doing this, I suggest you could consider using the VS's Scaffold feature.

It provides a way to help you integrate the identity with your existing projects.

If you don't have the dbcontext, you could directly new one.

Then it will help you set all the things, and set the connectiosntring with the SQL Server Express LocalDB.

More details, you could refer to this article.

This article, provide two solutions, one for the Scaffold Identity into an MVC project without existing authorization, another is for the Scaffold Identity into an MVC project with existing authorization.

After the project finish Scaffolding, you could run the migration follow the article shows using Visual Studio Connected Services or CLI.