Well, it turns out there this tiny little dropdown on the bottom of the debugger
Use it to switch from hex to utf to base64