I've recently run into the same issue and ended up using crossbeam::atomic::AtomicCell. On my x86-64 linux, AtomicCell<Instant> has the same size as Instant meaning it is lock-free, so no mutex overhead.