Is token refresh not taken care of internally? Is there anything extra we need to do here?

As per documentation

The DefaultAzureCredential class caches the token in memory and retrieves it from Microsoft Entra ID just before expiration. You don't need any custom code to refresh the token.

This is but present in System.Data.SqlClient (on .NET Framework) where, in certain scenarios, when a token expires for a connection in a connection pool, SqlClient can fail to discard the connection and refresh the token

Use Microsoft.Data.SqlClient this client often handles token expiration and Managed Identity better, with improved support for AAD token refresh. To resolve the issue in your code, add this NuGet package:

Install-Package Microsoft.Data.SqlClient -Version 5.1.0

After installing, update your code to use Microsoft.Data.SqlClient

please check this document for more information.