Based on Tsyvarev's suggestion I tried the LINK_DEPENDS and since I only use ninja, it works for me:
set_target_properties(${target} PROPERTIES LINK_DEPENDS ${CERTIFICATE_FILE})
where ${target} is every executable I need to sign