To remove using "Remove-Item", specify the path using "device paths", "\\?\" or "\\.\".

To remove the entire directory:

Remove-Item -Path "\\?\C:\Windows\SoftwareDistribution\Download\"

To remove only the subdirectories and keep the parent directory:

Remove-Item -Path "\\?\C:\Windows\SoftwareDistribution\Download\*"

The "\\?\" is supported in all versions of .NET Core and .NET 5+ and in .NET Framework starting with version 4.6.2.