For anyone interested, the solution was actually quite simple, basically, I added the sql command as my friend Erwin suggested.

def upgrade() -> None: op.execute("ALTER TABLE aluno ALTER COLUMN serie_aluno TYPE integer USING serie_aluno::text::int, ADD CONSTRAINT serie_aluno_serie_fkey FOREIGN KEY (serie_aluno) REFERENCES serie(id_serie);")