A thing I did was pickled the BM25Retriever.from_documents() at indexing time not run time. I'm not sure if it's the right decision, but let me know!