This is not really a solution, as I believe an issue exists with the libraries currently, but I found a workaround, by basically saving short videos with libcamera which turns out to be functional, and then parsing that as input for analysis for the opencv