Skip to content

Build precision 19 decimal scalars at i128 - #10903

Merged
connortsui20 merged 1 commit into
developfrom
ct/decimal-scalar-factory-width
Oct 28, 2026
Merged

connortsui20 merged 1 commit into
developfrom
ct/decimal-scalar-factory-width

Conversation

@connortsui20

Copy link
Copy Markdown
Member

Summary

vx.scalar(decimal.Decimal("9999999999999999999")) fails with an overflow error, and so does a JNI decimal literal at precision 19 with a value above i64::MAX. Both factories picked the value width from DecimalDType::required_bit_width(), which counts magnitude bits without a sign bit. At precision 19 that is 64, so they chose i64, which can't hold the largest 19-digit values.

Changes

Both factories now match on DecimalDType::canonical_values_type() from #10897, which gives i128 at precision 19. Every other precision keeps its width, because the sign bit still fits there. The Python precision inference test and a new JNI unit test cover the largest and smallest values at precision 19.

This is stacked on #10897 and retargets to develop after that merges.

Comment on lines +614 to +615
/// The [`DecimalValue`] variant of the canonical width of `dtype`'s precision.
fn canonical_decimal_value(value: i256, dtype: &DecimalDType) -> PyResult<DecimalValue> {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might blow things up but I think that is fine because it is a scalar, not an array?

@connortsui20
connortsui20 force-pushed the ct/decimal-scalar-factory-width branch from 81cc7b1 to 41073b9 Compare October 27, 2026 22:18
@codspeed

codspeed Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 10.34%

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 2 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

❌ 2 regressed benchmarks
✅ 2222 untouched benchmarks
⏩ 370 skipped benchmarks1
🗄️ 30 archived benchmarks run2

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Walltime threshold_avx2[80] 2.6 µs 2.9 µs -10.48%
❌ Walltime lt_i64_nullable_avx512 7.3 µs 8.2 µs -10.2%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ct/decimal-scalar-factory-width (c8ed4d5) with develop (55dca37)

Open in CodSpeed

Footnotes

  1. 370 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. 30 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them. ↩

@connortsui20
connortsui20 enabled stacked auto-merge (squash) October 28, 2026 12:37
stacked auto-merge was automatically disabled October 28, 2026 15:59

Pull request removed from stack

@connortsui20
connortsui20 removed this pull request from stack #10904 October 28, 2026 15:59
@connortsui20
connortsui20 added this pull request to stack #10932 October 28, 2026 15:59
@connortsui20
connortsui20 enabled stacked auto-merge (squash) October 28, 2026 16:01
@connortsui20
connortsui20 disabled stacked auto-merge October 28, 2026 16:02
Base automatically changed from ct/decimal-width-owner to develop October 28, 2026 16:02
The Python and JNI decimal scalar factories picked the value width from
`DecimalDType::required_bit_width()`, which counts magnitude bits without a
sign bit. At precision 19 it returns 64, so both factories chose `i64`, and a
valid value above `i64::MAX` failed with an overflow error. For example,
`vx.scalar(decimal.Decimal("9999999999999999999"))` failed.

Both factories now match on `DecimalDType::canonical_values_type()`, which
gives `i128` at precision 19. Every other precision keeps its width.

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20
connortsui20 force-pushed the ct/decimal-scalar-factory-width branch from 41073b9 to c8ed4d5 Compare October 28, 2026 16:03
@connortsui20
connortsui20 merged commit 565f090 into develop Oct 28, 2026
110 of 112 checks passed
@connortsui20
connortsui20 deleted the ct/decimal-scalar-factory-width branch October 28, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants