It's not clear what you mean with columns 10, 11, 12, and 13. While I'm guessing 10-11 default to 0 and increment, columns 12-13 are unknowns. Please provide a starting set, see How to make a great R reproducible example , [mcve], and https://stackoverflow.com/tags/r/info for how to share sample data. BTW, we do not need a lot of data, perhaps a few rows for 2-3 different IDs. Once we know the structure, we can scale it up arbitrarily to address the performance issues you think you are facing.