79596430

Date: 2025-04-28 12:06:59
Score: 4.5
Natty:
Report link

If I understand correctly, the float data type (which defaults to float (53)) can only store 17 digits.

Whereas '9 to the power 28' requires 27 digits.

So must I assume that the trailing 10 digits are just simply truncated off, and this is what leads to the inaccuracy?

So instead of working with

523347633027360537213511521

mssql is working with

523347633027360530000000000 ?

Reasons:
  • No code block (0.5):
  • Ends in question mark (2):
  • Self-answer (0.5):
  • Filler text (0.5): 0000000000
  • Low reputation (1):
Posted by: Joe Boyle