In VSCode the language is automatically detected. See in the bottom right corner, it might be written as JS, React or mdx. Click it → choose Configure File Association for '.mdx'… → set it to Markdown.
[Image showcasing file format chosen as markdown in the bottom-right corner of VSCode ↗️]