Hidden imports are not visible to the pyinstaller.

This function implicitly imports modules. A class named ONNXMiniLM_L6_V2 from one of these modules. This class also uses importlib.import_module to import such dependencies as "onnxruntime", "tokenizers", and "tqdm".

So we need to deal with all these imports.

The issue with hidden imports

To reproduce this error, we need a minimal project for pyinstaller.

Environment:

• Windows 10
• Python 3.11.9
• chromadb 0.6.3
• pyinstaller 6.12.0

Project sructure:

somedir\
  env # virtual environment directory
  pyinst # directory for pyinstaler files
  embedding.py # additional file for --onefile
  main.py

Python files:

embedding.py

def embedding_function():
    return "Hello"

main.py

import tkinter as tk

import chromadb

from embedding import embedding_function


root = tk.Tk()
label = tk.Label(root, text=embedding_function())
label.pack()
root.mainloop()

Steps to reproduce:

1. Create the Python files and the pyinst directory in some directory.

2. Run the following in cmd.

Volume:\somedir>python -m venv env

Volume:\somedir>env\scripts\activate

(env) Volume:\somedir>python -m pip install chromadb
...

(env) Volume:\somedir>python -m pip install pyinstaller
...

(env) Volume:\somedir>cd pyinst

(env) Volume:\somedir\pyinst>python -m PyInstaller "Volume:\somedir\main.py" --onefile -w
...

Pyinstaller will generate the necessary directories and the main.spec file.

pyinst\
  build # directory
  dist # directory, main.exe here
  main.spec

When I try to run main.exe I get the same error NameError: name 'ONNXMiniLM_L6_V2' is not defined.

Solutions

Hook and spec file

At this point, we need to create a hook for chromadb and edit the spec file to handle hidden imports.

1. Create a hook file and save it to the desired location.

hook-chromadb.py

from PyInstaller.utils.hooks import collect_submodules


# --collect-submodules
sm = collect_submodules('chromadb')

hiddenimports = [*sm]

2. Edit the spec file to add all the dependencies mentioned above.

Edit hiddenimports (--hidden-import) and hookspath (--additional-hooks-dir).

# -*- mode: python ; coding: utf-8 -*-


a = Analysis(
    ['Volume:\\somedir\\main.py'],
    pathex=[],
    binaries=[],
    datas=[],
    hiddenimports=['onnxruntime', 'tokenizers', 'tqdm'],
    hookspath=['Volume:\\path to the directory where the hook file is located'],
    hooksconfig={},
    runtime_hooks=[],
    excludes=[],
    noarchive=False,
    optimize=0,
)
pyz = PYZ(a.pure)

exe = EXE(
    pyz,
    a.scripts,
    a.binaries,
    a.datas,
    [],
    name='main',
    debug=False,
    bootloader_ignore_signals=False,
    strip=False,
    upx=True,
    upx_exclude=[],
    runtime_tmpdir=None,
    console=False,
    disable_windowed_traceback=False,
    argv_emulation=False,
    target_arch=None,
    codesign_identity=None,
    entitlements_file=None,
)

Run pyinstaller:

(env) Volume:\somedir\pyinst>python -m PyInstaller main.spec --clean
...

Now I can run main.exe without errors and see the root window.

CLI example

We do the same thing.

(env) Volume:\somedir\pyinst>python -m PyInstaller "Volume:\somedir\main.py" --onefile -w --collect-submodules chromadb --hidden-import onnxruntime --hidden-import tokenizers --hidden-import tqdm --clean

The generated spec file in this case:

# -*- mode: python ; coding: utf-8 -*-
from PyInstaller.utils.hooks import collect_submodules

hiddenimports = ['onnxruntime', 'tokenizers', 'tqdm']
hiddenimports += collect_submodules('chromadb')


a = Analysis(
    ['Volume:\\somedir\\main.py'],
    pathex=[],
    binaries=[],
    datas=[],
    hiddenimports=hiddenimports,
    hookspath=[],
    hooksconfig={},
    runtime_hooks=[],
    excludes=[],
    noarchive=False,
    optimize=0,
)
pyz = PYZ(a.pure)

exe = EXE(
    pyz,
    a.scripts,
    a.binaries,
    a.datas,
    [],
    name='main',
    debug=False,
    bootloader_ignore_signals=False,
    strip=False,
    upx=True,
    upx_exclude=[],
    runtime_tmpdir=None,
    console=False,
    disable_windowed_traceback=False,
    argv_emulation=False,
    target_arch=None,
    codesign_identity=None,
    entitlements_file=None,
)

You can do the same for other dependencies if they have hidden imports.