Skip to content

fix(numeric): replace decompose_float with stdlib formatting in format_float#6366

Closed
Chronolapse411 wants to merge 1 commit intotursodatabase:mainfrom
Chronolapse411:fix/format-float-precision
Closed

fix(numeric): replace decompose_float with stdlib formatting in format_float#6366
Chronolapse411 wants to merge 1 commit intotursodatabase:mainfrom
Chronolapse411:fix/format-float-precision

Conversation

@Chronolapse411
Copy link
Copy Markdown

Summary

Fix data corruption in float-to-text conversion by replacing the custom decompose_float digit-extraction algorithm with Rust's standard library formatting.

Fixes #6365

Problem

format_float() in core/numeric/mod.rs uses a custom decompose_float algorithm based on DoubleDouble arithmetic for extracting decimal digits from floating-point numbers. This algorithm produces different last-digit rounding than SQLite's internal sqlite3FpDecode, which uses long double / 80-bit extended precision.

When Turso formats a float (e.g., during text affinity conversion via CAST(x AS TEXT)), the resulting string can differ from what SQLite produces:

Input Turso (before) SQLite
-8487739174.3030205 -8487739174.3030205 -8487739174.30302
9.93e-322 9.93071948140905e-322 9.93071948140906e-322

When these differently-formatted strings are re-parsed into floats, the resulting bit patterns differ (up to 7 ULPs), causing differential test failures and real data corruption.

Fix

Replace decompose_float-based digit extraction with Rust's format!("{:.14e}"), which uses industry-standard Grisu3/Dragon4 algorithms. These produce the same digit sequences as SQLite's sqlite3FpDecode and C's printf("%.15g").

The new format_float implementation:

  1. Uses format!("{:.14e}", v) to get 15 significant digits
  2. Applies %g-style fixed vs. scientific notation switch (exponent range [-4, 14])
  3. Strips trailing zeros while preserving .0 for integer-valued floats
  4. Handles NaN, ±Inf, and 0.0 edge cases

Also removes the now-unused FloatParts enum and decompose_float function (dead code cleanup).

Verification

  • 1504 turso_core unit tests pass (0 failures)
  • clippy passes with -D warnings (0 warnings)
  • ✅ 3 new float formatting tests added (test_decode_float, test_format_float_matches_sqlite, test_format_float_edge_cases)
  • Differential simulator seed 6017 (previously failing): simulation succeeded
  • Seed sweep 6010–6025: 0 differential mismatches (6 PASS, 10 timeout-only)

…t_float

The custom decompose_float algorithm used DoubleDouble arithmetic for digit
extraction, which produced different last-digit rounding than SQLite's
sqlite3FpDecode. This caused data corruption when formatted floats were
re-parsed during text affinity conversion (e.g., CAST to TEXT).

Replace with Rust's standard library formatting (Grisu3/Dragon4), which
matches the digit sequences produced by SQLite's internal float-to-text
conversion. The fix changes format_float and format_float_scientific to
use format!("{:.14e}") for 15-digit precision digit extraction, then
reconstructs the %.15g-style output (fixed vs scientific notation switch
at exponent boundaries [-4, 14]).

Verified with:
- All 1504 turso_core unit tests pass
- Differential simulator seed 6017 (previously failing): PASS
- Seed sweep 6010-6025: 0 differential mismatches
Copy link
Copy Markdown

@turso-bot turso-bot bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please review @pereman2

@github-actions github-actions bot added the core label Apr 11, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Fossier: PR Auto-Closed

This PR was automatically closed because @chronolapse411 did not meet the trust threshold for this repository.

Score Breakdown

Total Score: 36.4/100 | Confidence: 100% | Outcome: DENY

Signal Value Score Weight
account_age 323 0.89 0.07
public_repos 28 1.00 0.04
contribution_history 28 0.14 0.04
follower_ratio 0.0 0.00 0.04
bot_signals False 0.50 0.03
open_prs_elsewhere 17 0.00 0.07
closed_prs_elsewhere 13 0.00 0.08
merged_prs_elsewhere 7 1.00 0.06
prior_interaction 0 0.00 0.24
activity_velocity 3 0.50 0.06
pr_content ... 1.00 0.06
commit_email no_email 0.50 0.03
pr_description ... 0.40 0.04
repo_stars 18180 0.30 0.03
org_membership 0 0.20 0.02
commit_verification ... 1.00 0.03
contributor_stars 0 0.00 0.03

Appeal

If you believe this is a mistake, please open an issue to request manual review. A maintainer can vouch for you by adding your username to the VOUCHED.td file.

You can also reach the maintainers at: https://discord.gg/mNUkCHs2R to appeal this decision

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq bot commented Apr 11, 2026

Merging this PR will degrade performance by 48.47%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
❌ 8 regressed benchmarks
✅ 300 untouched benchmarks
⏩ 105 skipped benchmarks1

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation format_float_infinity 1,172.8 ns 984.4 ns +19.13%
Simulation format_float_integer 5.3 µs 10.3 µs -48.47%
Simulation format_float_neg_infinity 1,201.1 ns 954.4 ns +25.84%
Simulation format_float_negative 5.4 µs 7.6 µs -29.55%
Simulation format_float_precision_edge 5.2 µs 7.6 µs -31.94%
Simulation format_float_very_small 6.3 µs 9.1 µs -30.7%
Simulation format_float_simple 5.3 µs 7.6 µs -29.67%
Simulation format_float_scientific_needed 7.1 µs 9.1 µs -21.97%
Simulation format_float_zero 1,107.2 ns 859.2 ns +28.87%
Simulation format_float_very_large 6.2 µs 9.2 µs -32.54%
Simulation length_float 7.2 µs 9 µs -20.46%

Comparing Chronolapse411:fix/format-float-precision (b71b0ba) with main (9b3dc16)

Open in CodSpeed

Footnotes

  1. 105 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: decompose_float produces incorrect rounding vs SQLite (data corruption in differential testing)

1 participant