Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pow overflows #668

Open
rawhuul opened this issue Aug 13, 2024 · 1 comment
Open

Pow overflows #668

rawhuul opened this issue Aug 13, 2024 · 1 comment

Comments

@rawhuul
Copy link

rawhuul commented Aug 13, 2024

Hi there, I am trying to calculate sigmoid of &[Decimal] here, which fails while using Decimal but works fine after being converted to f64, is this expected behaviour or a bug?

const E: f64 = 2.71828182845904523536028747135266250_f64; // 2.7182818284590451f64

let euler = Decimal::from_i128_with_scale(27182818284590451, 16);

let sgm = |d: &Decimal| -> Decimal { Decimal::ONE / (Decimal::ONE + euler.powd(-*d)) };

let sgf = |d: &Decimal| -> Decimal {
  let f: f64 = (*d).try_into().unwrap();
  Decimal::from_f64(1.0 / (1.0 + f64::powf(E, -f))).unwrap()
};
          
self.iter().map(sgf).collect()
# Output while using closure sgm
thread 'tests::test_float64data' panicked at ~\.cargo\registry\src\index.crates.io-6f17d22bba15001f\rust_decimal-1.35.0\src\maths.rs:290:21:
Pow overflowed

# Output while using function: sfm
[0.999993855825398, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 1, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 1, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 1, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 0.999999999897381, 0.999993855825398, 1, 0.731058578630005, 0.731058578630005, 0.731058578630005, 0.731058578630005, 0.731058578630005, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.880797077977882]
@paupino
Copy link
Owner

paupino commented Aug 14, 2024

This is unfortunately related to this issue: #431

The long story short is that the way that powd works is it tries to calculate the exponent and then reduce the number back to the correct scale (rounding if necessary). Currently this uses multiplication to derive this - of which behind the scenes multiplication temporarily expands the decimal to 192 bits before shrinking back to 96 bits. Because of this, every "iteration" within powd effectively expands and then shrinks - which of course is not effective and causes the overflow.

What should happen is that the calculation should expand to 192 bits (or more) but not scale back to 96 bits until the very last opportunity. This would help resolve situations like you're seeing above.

Overall, the fix isn't difficult, however requires some critical path refactoring to allow for more lenient multiplication steps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants