In 2020 I needed a portable binary of OpenSSL for Windows, so I compiled it with Cygwin and made the binaries available. If you don't need the GOST (Russian Algorithms) libraries, download OpenSSL-1.1.1h_win32(static)[No-GOST].zip. It's a static binary (doesn't depend on dynamic DLL libraries), portable to have on a USB stick. No installer, Open Source Excellence.
https://sourceforge.net/projects/openssl-for-windows/