If you didn't change the number of shards, then deduplication should work eventually and you should always insert data into the same shard

Did you use something like that

SELECT shardNum(), id, count() c FROM cluster('my-cluster',my_db.my_table_local) FINAL GROUP BY ALL HAVING c > 1

to check deduplication works or use another approach?