The accepted answer by TiMeJuMp works, but I had to install the German language pack first via:

Install-Language de-DE

Unfortunately Windows 10 does not seem to warn about the missing language pack when trying to change the language without it.

See also: https://learn.microsoft.com/en-us/powershell/module/languagepackmanagement/install-language