On the magit status page, type d for diff, r for range, and then enter "master" for the branch to diff with.

It will show all the diffs, the trick is, on the Magit-diff buffer press Shfit+Tab (which is bind to "magit-section-cycle-global") to collapse the sections and show only the file names.