Turns out the issue was that, I was using only one subtable. Each subtable get computed once so if multiple pairings are in there it doesnt work. The solution was to create more GPOS subtables and split the pairings accordingly.