Turns out the UNION ALL query wasn't running originally as fast as I thought, it did run eventually.
Thanks all for your help