My solution was to use
target_link_options(${TARGET_NAME} PRIVATE "/PDBALTPATH:$<TARGET_PROPERTY:${TARGET_NAME},NAME>.pdb")
To specify the link options on the target to use the full path to the pdb file.