diff options
| author | Yusuke Endoh <[email protected]> | 2025-04-06 18:36:06 +0900 |
|---|---|---|
| committer | Yusuke Endoh <[email protected]> | 2025-04-07 11:08:10 +0900 |
| commit | 3a7b9ca93b91dcc086b9ac8b9957e59268f9493b (patch) | |
| tree | b129fb6d901cb8baa05031e67db38ae201883dd3 /numeric.c | |
| parent | e25889951f39aff6e3c16ecee10e678912454e69 (diff) | |
Fix `Integer.sqrt` to never exceed actual value
`Integer.sqrt` uses `sqrt(3)` from libm for small values.
This method must return a value less than or equal to the actual integer
square root, but libm's sqrt does not always guarantee that.
This change corrects that by decrementing the result if necessary.
Fixes [Bug #21217]
Notes
Notes:
Merged: https://github.com/ruby/ruby/pull/13076
Diffstat (limited to 'numeric.c')
| -rw-r--r-- | numeric.c | 6 |
1 files changed, 5 insertions, 1 deletions
@@ -5978,7 +5978,11 @@ prefix##_isqrt(argtype n) \ while ((t = n/x) < (argtype)x) x = (rettype)((x + t) >> 1); \ return x; \ } \ - return (rettype)sqrt(argtype##_TO_DOUBLE(n)); \ + rettype x = (rettype)sqrt(argtype##_TO_DOUBLE(n)); \ + /* libm sqrt may returns a larger approximation than actual. */ \ + /* Our isqrt always returns a smaller approximation. */ \ + if (x * x > n) x--; \ + return x; \ } #if SIZEOF_LONG*CHAR_BIT > DBL_MANT_DIG |