This is an issue with not having readline and xz installed. Documentation here gives you the suggested build libraries. Once you have installed the libraries you can reinstall your preferred version of python (e.g. pyenv install 3.12.8) and you should have working arrow keys and all the other fun stuff from readline.