Thanks for this solution!
My idea was similar to @Kek Silva:
$user = [System.Security.Principal.WindowsIdentity]::GetCurrent().Name
$regRights = [System.Security.AccessControl.RegistryRights]::SetValue
$inhFlags = [System.Security.AccessControl.InheritanceFlags]::None
$prFlags = [System.Security.AccessControl.PropagationFlags]::None
$acType = [System.Security.AccessControl.AccessControlType]::Deny
$rule = New-Object System.Security.AccessControl.RegistryAccessRule($user, $regRights, $inhFlags, $prFlags, $acType)
$acl = Get-Acl 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.log\UserChoice2'
$acl.RemoveAccessRule($rule)
$acl | Set-Acl -Path 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.log\UserChoice'
Remove-Item -Path HKCU:\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.log\UserChoice -Confirm:$false
Can someone explain why the typical approach doesn't work?