Got it myself. It was me renaming some .py files, and not deleting pickle-based cache file. Old module names got stuck in file, and pickle tried to import those old module names which no longer exist.