By adding Distinct and Union, I was able to satisfy EF Core.
Instead of query.Concat(query), I used query.Distinct().Concat(query.Distinct()). You might not need Distinct on both branches. See the comment below for more info.
https://github.com/dotnet/efcore/issues/32046#issuecomment-3393373171