I found that using the command that another user mentioned works well, but instead of using xxd I find it better to use od since it seems to be pre-installed on linux.

code --folder-uri vscode-remote://attached-container+$(printf "$CONTAINER_NAME_OR_ID" | od -An -tx1 | tr -d ' \n')

Edit: I created a new answer instead of commenting because I don't have enough reputation to comment.