Yes, in "online" case please downgrade, but there is one more trick, just remove unwanted version file while offline to bind "head" to "base" back. For example, occasionally I've created revision, and then remove it before alembic upgrade
alembic upgrade