Answering my own question here. The issue was adjusting the cursor offset based on the font baseline. The default baseline is at the top, so a 0,0 works fine, but others align to the bottom, meaning you need to "push" it back onto the display. Thx!