79517456

Date: 2025-03-18 12:45:26
Score: 1.5
Natty:
Report link

Out of 17 million records, what is the count of distinct keys. How many are going for change ?

Option 1:

Break sq results in 10 parts using mod function.

If target partition is not possible, try source partition. Create 10 session on existing mapping. In each session add a condition mod = 0, mod =1 like wise (sq overwrite). See if you can run all these sessions in parallel.

Option 2:

See if you can reduce number of updates. Update only when it is very absolutely necessary. 10 inserts are better then 1 update. Handle the SCD logic in mapping. And insert two records, Instead of update old record and insert new record. Update is a costly operation.

Option 3:

As Maciejg has suggested earlier. Drop unnecessary indexes and re-create in post sql.

Option 4:

Create 10 temp tables, insert and update each table using mod function. and finally insert into main table. For this we need 10 targets and router.

Good luck !!

Reasons:
  • Long answer (-0.5):
  • No code block (0.5):
  • Contains question mark (0.5):
  • Low reputation (1):
Posted by: chaitanya ch