I found that I setup the wrong offset of my color attribute.

posX posY posZ uvS uvT colorR | colorG colorB colorA

I setup the offset here so alpha value is read from the posX of the next vertice. So when the posX is negative, the alpha value is wrong.