Publishing a little update because I keep getting pinged by stackoverflow because people still face the same issue. Some answers are quite good because they explain me how to convert my current project into something that work. Thanks for your answers and efforts. But my question was really how to create a Python project where I can have my own custom package and use it easily.

So for the record, if it can help someone in the future, this is what I was looking for: https://docs.astral.sh/uv/concepts/projects/init/#packaged-applications

So you have to create a project with

uv init --package example-pkg

and then you can just do uv run <your-script> and your custom package will be referenced automatically. So if in your script you are doing a import example-pkg it will work. No need for any setup.py or anything else.