While searching around, I found this, which solved the issue.
The solution is:
sqlalchemy2-stubs
sqlalchemy-stubs
mypy.ini
plugins = sqlalchemy.ext.mypy.plugin
This solved the issue.