Looks like your server does not deal well with (int)client.MaxReadSize as minimum read size,

you may want to obtain the file size in advance, set remainingFileLength to it, and then use Math.Max((int)client.MaxReadSize, remainingFileLength)

for each read.