If you say that code is usually represented in latex, code-blocks or ticks. you have the punctuations which you might need to ignore at the time of tokenization. Use libraries specific to each syntax (e.g., pyparsing for Python, pylatex for LaTeX) to parse and clean code snippets and formulas.