You have to right-click the file to open the context menu and select Open with...

Then choose the "Text Editor" option.

You can also associate a default editor for svg files, either through the option in the previous menu, or via a setting in your config:

"workbench.editorAssociations": {
  "*.svg": "default"
}