The problem was with the ids repeating across several batches, which resulted in overwriting a part of the data, which had been previosly loaded