Turns out the error was elsewhere in my code - a hash table was only getting updated with the last label and associated data. So the issue was not with VoiceOver at all.