Thanks for the hint about StreamReader being text only....doh!

so creating a memorystream works great.

ms = new MemoryStream(client.DownloadData(baseUrl + endPoint));
Response.Clear();
Response.BufferOutput = true;
Response.ContentType = "application/pdf";
Response.AddHeader("content-disposition", "attachment; filename="report.pdf");
Response.BinaryWrite(ms.ToArray());

Thanks all