As it turns out, the solution was simpler than I expected.
Since those files are not necessary, I could simply remrove them from the repo and add them to .gitignore:
.gitignore ... venv/ __pycache__/