So my question: is it actually a unit test if it uses the real database or is it an integration test? Am I using repository pattern wrong since I cannot unit test it with mock or in-memory database?

The end goal of writing unit or integration tests is to allow you to confidently make changes (improvements) to your code as time goes by and at the same time to be relatively confident that these newly introduced changes don't break the existing functionality by running tests that correctly indicate if the system under test behaves as expected or not (Pass or Fail). And this should be achieved with no or minimal changes on the tests themselves since frequently amending tests most likely will lead to bugs or errors in the tests. This must be your main aim when testing your app not whether your tests are pure unit test. Pure unit tests e.g. testing all (or almost all) methods in isolation with each dependency mocked or stubbed out, are normally a lot fragile the smallest code changes lead to serious changes in the tests. This is somewhat opposite to the main goal of testing which is solid and stable tests that correctly indicate if something is broken and that don't provide you with a ton of false negative or false positive results. To achieve this the best way is to take a more higher level integration approach of testing your app (especially it it is an asp.net core web application with a database) e.g. not to mock your database repositories but rather than that use sql server localdb with pre seeded data in it.

For more insights on which is the correct testing approach you should follow when writing tests for web apps/web apis I strongly recommend you to read this article TDD is dead. Long live testing.

Just one quote from it

I rarely unit test in the traditional sense of the word, where all dependencies are mocked out, and thousands of tests can close in seconds. It just hasn't been a useful way of dealing with the testing of Rails applications. I test active record models directly, letting them hit the database, and through the use of fixtures. Then layered on top is currently a set of controller tests, but I'd much rather replace those with even higher level system tests through Capybara or similar.

and this is exactly how Microsoft recommends testing Web Apis with a database Testing against your production database system

public class TestDatabaseFixture
{
    private const string ConnectionString = @"Server=(localdb)\mssqllocaldb;Database=EFTestSample;Trusted_Connection=True;ConnectRetryCount=0";

    private static readonly object _lock = new();
    private static bool _databaseInitialized;

    public TestDatabaseFixture()
    {
        lock (_lock)
        {
            if (!_databaseInitialized)
            {
                using (var context = CreateContext())
                {
                    context.Database.EnsureDeleted();
                    context.Database.EnsureCreated();

                    context.AddRange(
                        new Blog { Name = "Blog1", Url = "http://blog1.com" },
                        new Blog { Name = "Blog2", Url = "http://blog2.com" });
                    context.SaveChanges();
                }

                _databaseInitialized = true;
            }
        }
    }

    public BloggingContext CreateContext()
        => new BloggingContext(
            new DbContextOptionsBuilder<BloggingContext>()
                .UseSqlServer(ConnectionString)
                .Options);
}

public class BloggingControllerTest : IClassFixture<TestDatabaseFixture>
{
    public BloggingControllerTest(TestDatabaseFixture fixture)
        => Fixture = fixture;

    public TestDatabaseFixture Fixture { get; }

[Fact]
public async Task GetBlog()
{
    using var context = Fixture.CreateContext();
    var controller = new BloggingController(context);

    var blog = (await controller.GetBlog("Blog2")).Value;

    Assert.Equal("http://blog2.com", blog.Url);
}

In short they use a LocalDB database instance, seed data in this instance using the test fixture and executing the tests on a higher integration level i.e. calling the controller method which calls a service(repository) method that queries the Blogs dbSet on the dbContext that executes a Sql query to LocalDB that returns the seeded data.