I cannot comment yet so i will be answering instead.

I have tried using Llava for Policies.pdf and stuff for RAG FAQ chatbot, however that multimodal model have also said the same thing to me, either the image is not good enough or it just takes a guess. It is able to understand images though, like dogs, or graphs (in my case it just tells me the image is a graph but cannot give details about it). Even when upscaled with pdf to image to upscaler, it still say the same thing, but can now identify more text on the image.

Ultimately to reach some weird deadline i threw that Model and implemented a pre-processing, PDF to Image to OCR. And yes it makes Llava redundant in here. But you can use other models to process the text output of OCR to reach the same "Document Understanding" task.

Another thing: PDF Documents are a pain, some of them may be all text and can be scraped easily with already available PDFpackages. But some may contain Images, or just all Scanned Images of the actual printed document. That is where Image to OCR will be of use to you. You can try other vision models or OCR packages but currently from all i've tested winocr works best for image text extraction.(yes, its using the OCR engine of windows snipping tool)