Skip to content

incorrect RoundUp conversion of large strings #176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
stevengj opened this issue Dec 11, 2023 · 2 comments
Open

incorrect RoundUp conversion of large strings #176

stevengj opened this issue Dec 11, 2023 · 2 comments
Labels

Comments

@stevengj
Copy link
Member

stevengj commented Dec 11, 2023

This is wrong, since the result should be exactly representable in Dec128:

julia> Dec128(big(10)^1000, RoundUp)
1.000000000000000000000000000000001e1000

Unfortunately, this might be a bug in the underlying library? All we are doing is converting the BigInt to a string and then calling the library's from_string routine.

As a result, we get incorrect results like:

julia> d128"1e100" == big(10)^100 # correct
true

julia> d128"1e1000" == big(10)^1000 # incorrect
false
@stevengj stevengj added the bug label Dec 11, 2023
@jmkuhn
Copy link
Collaborator

jmkuhn commented Dec 11, 2023

The results vary run to run. Most likely a buffer overflow in the C. I'll look into it.

julia> Dec128("1" * "0"^106, RoundUp)
1.000000000000000000000000000000001e106

julia> Dec128("1" * "0"^105, RoundUp)
1.0e105

julia> exit()
CD0004431999 1006% julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.4 (2023-11-14)
 _/ |\__'_|_|_|\__'_|  |  
|__/                   |

julia> using DecFP

julia> Dec128("1" * "0"^106, RoundUp)
1.0e106

@jmkuhn
Copy link
Collaborator

jmkuhn commented Feb 21, 2025

I have a patch for this one. I want to do a bit of testing on aarch64, musl, and perhaps riscv64 before releasing a new DecFP_jll.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants