I was created a boilerplate for integrating CodeIgniter 3 and Vue 3, and for the CodeIgniter 4 the idea is still the same.
You can found it here: https://github.com/ngekoding/codeigniter-vue-boilerplate
The main parts to make it works:
It is a basic configuration, I think you can easily fit it for your CodeIgniter 4.
Let me know if you still get any issue.