As you have correctly found, \underline, is not a currently supported MathText command. But, matplotlib's MathText is not the same a LaTeX. To instead use LaTeX, you can do, e.g.,

import matplotlib.pyplot as plt

# turn on use of LaTeX rather than MathText
plt.rcParams["text.usetex"] = True
plt.text(.5, .5, r'Some $\underline{underlined}$ text')
plt.show()

You may have issues if your tex distribution does not ship with the type1cm package, in which can you may want to look at, e.g., https://stackoverflow.com/a/37218925/1862861.