Mask for a directory have to end with a slash:
https://winscp.net/eng/docs/file_mask#directory
So like this:
| System Volume Information*/
Or */
, when excluding all directories.
See How do I transfer (or synchronize) directory non-recursively?