The precision issue arises from the limited precision of float. To minimize this error you could use double precision. If possible, you could use double for calculations and only convert to float when transferring to the GPU.

I also heard about Kahan summation, but I never actually used it so I can't tell you if this fully applies for your use case.