None of the above worked, so I removed the C:\Program FIles\Docker directory, the C:\ProgramData\Docker directory, the uninstall registry mentioned above and also the hkey_local_machine\software\docker. Rebooted and installed the latest version, that worked.
Note that this will erase any configuration or stored data.