I need some help on this one too. Trying to work on large file issues seems to hit the same problem: how to manage the internal buffering. Thanks for your references to UnmanagedMemoryStream; but MSFT still doesn't let you easily solve that core issue.

FileStream's loading options will reveal this problem very fast. When loading a large file, you'll be forced into MemoryMappedFile, or you won't have a solution at all (!!!)

Reading a 90MB text file loads approx 300MB of (managed) memory; and locks my rendering thread while I have a background worker trying to read these 90MB sequentially. It would otherwise be that you don't need to do that pre-loading. You may just be able to read each line with some native call. MSFT doesn't like this because they've tried too hard to manage everything on their backend - now with Task (TPL) thread support. So, there's "too much overhead"; but maybe not in each situation if we already know how to manage our access to large files, or handle threading. We can space out native calls, even, if you'd let us have access to the native file stream.

My current implementation uses and UnmanagedMemoryStream - which wants a SafeBuffer, or native pointer. However, there is no (native) call that I've found to ask for a portion of the file - nor is there a simple call to get data from a SafeFileHandle. So, I'm stuck parsing MSFT's FileStream code trying to figure out when it's safe to do.