Repository navigation
Decimal('0.0000005').quantize(Decimal('1.1111111')) -> Decimal('5E-7') # should be unchanged #128373
Closed as not planned
Description
Bug report
Bug description:
import decimal
Decimal = decimal.Decimal
Decimal('0.000005').quantize(Decimal('1.111111')) # Decimal('0.000005') correct
Decimal('0.0000005').quantize(Decimal('1.1111111')) # Decimal('5E-7') wrongAccording to the documentation, https://docs.python.org/3/library/decimal.html#decimal.Decimal.quantize
quantize() should "Return a value equal to the first operand after rounding and having the exponent of the second operand."
Clearly, it is not respecting the exponent of the second operand, even if that exponent is given explicitly:
Decimal('0.0000005').quantize(Decimal('1.1111111e0')) # Decimal('5E-7') wrongCPython versions tested on:
3.13
Operating systems tested on:
Linux
Activity
michael-db commentedon Dec 31, 2024
More directly, the immediate problem is
But there is already a normalize() function for that if I wanted it
So, in this case of leading zeroes, normalize() has been rendered superfluous, while quantize() has been rendered useless.
If this limitation is not to be corrected then at a minimum it needs to be documented.
skirpichev commentedon Dec 31, 2024
I suspect you misunderstand which "exponent" referenced in docs here, assuming it's just zero in both cases. It's not. You can use as_tuple() method to check exponent value. Then, results seems correct for me:
So, I don't think there is an issue.
Edit: For me it seems that from the documentation is clear which exponent referenced (and it's not one, shown in repr output or provided by you in string form of the Decimal constructor). So, I doubt there is a documentation issue as well.
What do you mean by "that"? The normalize() function is to get the canonical representation for a Decimal.
michael-db commentedon Dec 31, 2024
"No issue"? If the exponent referred to is the internal exponent, not that appearing in the representation, then the documentation at least needs to make that clear. The example doesn't, and I am not the first person to find this quite misleading https://stackoverflow.com/questions/61704496/ Documentation is supposed to be for those unfamiliar with a module, not those who already know.
Like the person asking that question, I wanted control over the representation. The normalize() and quantize() methods appear to provide that control within the decimal module, and would be more useful if they did.
skirpichev commentedon Dec 31, 2024
It's not "internal", this value is available via public interface. And you also can construct a Decimal object from tuple.
Just don't mix the exponent value for a Decimal object with it's string representation. Current documentation seems clear for me in this aspect.
You can take look on
_pydecimal.Decimal.__str__method to see how the exponent value used to form a string representation. Fixed point or scientific notation chosen based on the exponent value. But if the number formatted in a fixed point notation - that doesn't mean it has zero exponent.It's not clear what do you mean. Probably, you want to control a string representation. In this case you should look on string formatting (
str.format()method and f-strings).michael-db commentedon Dec 31, 2024
Thank you for your reply, but I must say it again: Documentation is supposed to be for those unfamiliar with a module, not those who already know.
An example making clear that the exponent referred to was the internal one, together with a link to the as_tuple() method for what is meant by "exponent" would solve the problem as far as the documentation is concerned.
That would be more productive than arguing about words like "internal" (what you call it is entirely irrelevant, the fact is that it is a distinct exponent from the representative string) and disregarding the evidence that those unfamiliar with the module have found the documentation misleading on this point.
skirpichev commentedon Dec 31, 2024
Again, it's not "internal". It's the exponent value, referred in many other places across the docs. You mistakenly interpreted the "exponent" here as "the exponent in scientific notation, when some Decimal represented this way in a string form". Could you please explain how did you come to this interpretation from the docs?
There is a link to the decimal arithmetic specification, where all terms are defined. If you are unfamiliar with the floating-point arithmetic (not with the module!) - it's a good source to start with. Or just read Wikipedia article: https://en.wikipedia.org/wiki/Floating-point_arithmetic
Unfortunately, this "problem" span many other methods. I don't think we should clutter documentation with examples, that should be obvious to a careful reader.
IMO, sphinx docs for the decimal module already have examples, showing the difference between string representation and exponent notion. For instance, from constructor docs:
But lets keep this for a while, maybe someone else will see how docs should be improved.
michael-db commentedon Dec 31, 2024
If you like Wikipedia, try reading this page about a certain cognitive bias in evidence here: https://en.wikipedia.org/wiki/Curse_of_knowledge It explains why, as a general principle, you should value feedback from reviewers instead of blaming them for being careless.
On a "careful reading", an error in the documentation for to_eng_string() would be obvious, yet you have not spotted it. Perhaps that example could lead you to have a little more empathy for readers. It is also one of several places where the word "exponent" is used to mean exactly what people expect it to mean, i.e., whatever comes after the 'E'.
Clutter: This is a poor excuse. With a little imagination, no clutter is required. If you really need to limit yourself to a single example, you could choose one that avoids misunderstanding. If you need to use the word "exponent" to refer to two different things, then you could add a one-word qualifier to avoid misinterpretation. If you have a peculiar objection to the word "internal" then make up another one and define it.
Arguing with someone whose default response is to put all their effort into denying that there is a problem and making excuses for not fixing it is not a good use of my time. I won't be commenting further.
skirpichev commentedon Dec 31, 2024
Not at all. But it's enough to provide you some basic information about floating-point arithmetic.
If you found an error in docs - feel free to open a bug.
I don't sure you are right about people expectations. People, who read something such basic as https://en.wikipedia.org/wiki/Floating-point_arithmetic - will assume very different meaning for exponent in context of floating-point arithmetic.
Are other places also related to string representation of Decimal's?
Can you propose some concrete suggestion?
terryjreedy commentedon Dec 31, 2024
@rhettinger
rhettinger commentedon Jan 2, 2025
ISTM that the docs appropriately mirror the wording in the specification for quantize.
Likewise, the intro section of the decimal docs do a reasonably good job of explaining the concept that
Decimal('1.1111111')means11111111 × 10⁻⁷with-7being the exponent — not as an internal concept but as the core concept behind the entire decimal arithmetic specification. The intro section also includes two typical examples of how to usequantize().Further along, the detailed docs for quantize give a useful and clear example demonstrating that the meaning of
Decimal('1.000')is1000 × 10⁻³with-3as the exponent.I recommend closing this as
not a bug. The current docs seem reasonable to me as well. Wording tweaks would take just us farther from the specification but not actually help a new reader with strong preconceptions and who may be starting in the middle of the module docs.skirpichev commentedon Jan 3, 2025
Not just decimal, but a floating-point.
Yep, I'll do this.
One point is that we could add to the glossary term like "floating-point" / "floating-point arithmetic" with some introduction for basic concepts (exponent, significand, etc) and then use it in some places across the docs. But I'm not sure if it worth.
mdickinson commentedon Jan 5, 2025
@skirpichev
That would be nice, but the reality is unfortunately messier.
The IEEE 754 standard uses two different conventions, switching between them as convenient. The value of the exponent for any given floating-point number depends on whether the significand is being interpreted as an integer or whether it's interpreted as having one bit (or digit) before the binary (or decimal) point and the remaining bits (or digits) following the point. Somewhat frustratingly, the standard just uses the word "exponent" in both cases without any qualifying adjective, but it disambiguates via the notation used: the text uses 'e' to denote an exponent for the digit+fraction significand interpretation, and 'q' for the exponent when the significand is interpreted as an integer. So for example for the IEEE 754 binary64 format that CPython now expects, the maximum "e" exponent is
1023, while the maximum "q" exponent is971.By way of example, here's
sys.float_info.maxrepresented using the two different conventions - first with an "e" exponent of 1023, then with a "q" exponent of 971:Here's the definition of "exponent" from section 2.1 of the IEEE 754 standard:
To add to the potential for confusion, the C standards (see e.g., §5.2.4.2.2 of the C17 standard) use a different convention again, interpreting all the bits (or digits) of the significand as lying to the right of the binary (or decimal) point. With that convention, the maximum exponent for IEEE 754 binary64 is
1024rather than1023. That's whysys.float_info.max_expis1024, rather than1023or971, and why the exponent returned bymath.frexpfor any given finite nonzero floatxis one larger than the exponent that would be returned by an implementation of IEEE 754'slogBoperation.It would be nice if there were standard language for describing the two (three, including the C standard) different meanings of exponent, but I'm not aware of anything widespread or standardised.
mdickinson commentedon Jan 5, 2025
FWIW, for the specific case of the (rather niche) standard that Python's
decimalmodule is based on, the wording "adjusted exponent" is used for what I've been calling the "e" exponent above (and theDecimal.adjusted()method gives direct access to that value), while "exponent" refers to the "q" exponent. But that use of "adjusted exponent" doesn't seem to be standard elsewhere.skirpichev commentedon Jan 6, 2025
It's true, but something that probably might be mentioned breefly we a link to more complete source (like referenced wikipedia article, which does mention this).
And the MPFR.
Maybe confusion is less possible for floats and instead we should just inline the relevant part of the specification or just link to that in the decimal intro section. E.g., right at beginning:
BTW, this is not the first bugreport I've seen, where people are trying to do strange things with Decimal's when they actually want some formatted string output. Docs lacks any examples (here too.). Maybe it worth to add something in the "quick-start" section? (Fixed-point/scientific, precision, effect of the context's rounding mode.)
ericvsmith commentedon Jan 9, 2025
Yes, I think this is a good idea.
skirpichev commentedon Jan 10, 2025
FYI: #128698 adds a reference to spec and some formatting examples.