The answer from @User and @Dawoodjee is correct, and I recommend it.
However, as explained in the updated docs, once you've connected to a folder initially, it should appear in a drop-down menu on the SSH extension sidebar as shown here. This allows for quicker access on future connections to the same remote folder.
Image retrieved from: https://github.com/microsoft/vscode-docs/blob/56c846422e796b0f50c655a67cfdd8fe68590d47/docs/remote/images/ssh/ssh-explorer-open-folder.png
