So I believe you are missing the gist here by mentioning it. Lazy loading simply means, nothing will be worked upon until compute() function is called, which will then take care of execution.!

So you need to have all your logic in place, at the end you can call compute function.

If your data is too large, compute internally merges everything and brings the entire data into the caller, which may still kill you application, die to OOM, so better to use map_partions with compute, to write it into separate files or push output to some database.!