Well I finally found the answer myself. I had to open a code window and click on the Tools menu. This revealed a list containing References... Macros... Options... and (wait for it) Oldname Properties... and when I clicked on this last one it opened a rather boring page with a number of text boxes, the first one being labelled Project Name. This contained the old name, so I entered the new name.

Seemed a rather silly place to hide this information, but then it is a Microsoft Application so one shouldn't be too surprised.

By the way, when I clicked on the link in the email I received from Stack Overflow, it led me to a login page where none of my saved passwords worked. Eventually I tried going to https://stackoverflow.com and found I was automatically logged in straight away! I'll never understand web security...