This is a classic stream / buffer reading issue. Instead of bulk processing the entire streaming response from Solr, the intention is that the client will read it chunk by chunk and make sense of it as you go.

I have not tried this myself but there are streaming json parsers for the Python ecosystem (https://pypi.org/project/json-stream/) which seems to fit the bill at a glance. I believe you will be able to configure it so that your code consumes each doc at a time while still reading the stream from your streaming request.

Good luck

-- Jan Høydahl - Apache Solr committer