Skip to content

fix(sqlite): allow 32766 query params - #4810

Open
Weakky wants to merge 2 commits into
mainfrom
fix/sqlite_max_bind_values
Open

fix(sqlite): allow 32766 query params#4810
Weakky wants to merge 2 commits into
mainfrom
fix/sqlite_max_bind_values

Conversation

@Weakky

@Weakky Weakky commented Apr 4, 2024

Copy link
Copy Markdown
Contributor

Overview

closes https://github.com/prisma/team-orm/issues/1048

Since SQLite 3.32.0, the max bind values were upgraded from 999 to 32766. As of commiting is, we're bundling SQLite 3.41.1.

This will improve performance for all queries that had more than 999 query params.

@Weakky
Weakky requested a review from a team as a code owner April 4, 2024 16:02
@Weakky
Weakky requested review from jkomyno and removed request for a team April 4, 2024 16:02
@github-actions

github-actions Bot commented Apr 4, 2024

Copy link
Copy Markdown
Contributor

Wasm Query Compiler File Size

Engine This PR Base branch Diff
Postgres 3.508MiB 3.508MiB 0.000B
Postgres (gzip) 1.143MiB 1.143MiB 0.000B
Postgres (size-optimized) 1.767MiB 1.767MiB 0.000B
Postgres (size-optimized, gzip) 695.608KiB 695.607KiB 1.000B
Mysql 3.461MiB 3.461MiB 0.000B
Mysql (gzip) 1.128MiB 1.128MiB 2.000B
Mysql (size-optimized) 1.737MiB 1.737MiB 0.000B
Mysql (size-optimized, gzip) 684.341KiB 684.338KiB 3.000B
Sqlite 3.378MiB 3.378MiB 1.000B
Sqlite (gzip) 1.099MiB 1.099MiB 2.000B
Sqlite (size-optimized) 1.693MiB 1.693MiB 1.000B
Sqlite (size-optimized, gzip) 667.395KiB 667.392KiB 3.000B
SQL Server 3.596MiB 3.596MiB 0.000B
SQL Server (gzip) 1.161MiB 1.161MiB 0.000B
SQL Server (size-optimized) 1.766MiB 1.766MiB 0.000B
SQL Server (size-optimized, gzip) 698.452KiB 698.451KiB 1.000B
CockroachDB 3.558MiB 3.558MiB 0.000B
CockroachDB (gzip) 1.162MiB 1.162MiB 70.000B
CockroachDB (size-optimized) 1.792MiB 1.792MiB 0.000B
CockroachDB (size-optimized, gzip) 705.556KiB 705.557KiB -1.000B

@codspeed-hq

codspeed-hq Bot commented Apr 4, 2024

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 11 untouched benchmarks
⏩ 11 skipped benchmarks1


Comparing fix/sqlite_max_bind_values (3da6a45) with main (e922089)

Open in CodSpeed

Footnotes

  1. 11 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@github-actions

github-actions Bot commented Apr 4, 2024

Copy link
Copy Markdown
Contributor

✅ WASM query-engine performance won't change substantially (0.998x)

Full benchmark report
DATABASE_URL="postgresql://postgres:postgres@localhost:5432/bench?schema=imdb_bench&sslmode=disable" \
node --experimental-wasm-modules query-engine/driver-adapters/executor/dist/bench.mjs
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
"
cpu: AMD EPYC 7763 64-Core Processor
runtime: node v18.19.1 (x64-linux)

benchmark                   time (avg)             (min … max)       p75       p99      p999
-------------------------------------------------------------- -----------------------------
• movies.findMany() (all - ~50K)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     368 ms/iter       (364 ms … 374 ms)    372 ms    374 ms    374 ms
Web Assembly: Latest       453 ms/iter       (451 ms … 459 ms)    454 ms    459 ms    459 ms
Web Assembly: Current      452 ms/iter       (450 ms … 457 ms)    454 ms    457 ms    457 ms
Node API: Current          197 ms/iter       (193 ms … 210 ms)    200 ms    210 ms    210 ms

summary for movies.findMany() (all - ~50K)
  Web Assembly: Current
   2.3x slower than Node API: Current
   1.23x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movies.findMany({ take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  14'820 µs/iter (14'455 µs … 18'029 µs) 14'687 µs 18'029 µs 18'029 µs
Web Assembly: Latest    19'184 µs/iter (18'057 µs … 22'515 µs) 19'154 µs 22'515 µs 22'515 µs
Web Assembly: Current   18'460 µs/iter (18'266 µs … 18'701 µs) 18'555 µs 18'701 µs 18'701 µs
Node API: Current        8'121 µs/iter   (7'909 µs … 8'379 µs)  8'223 µs  8'379 µs  8'379 µs

summary for movies.findMany({ take: 2000 })
  Web Assembly: Current
   2.27x slower than Node API: Current
   1.25x slower than Web Assembly: Baseline
   1.04x faster than Web Assembly: Latest

• movies.findMany({ where: {...}, take: 2000 })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   2'336 µs/iter   (2'209 µs … 3'534 µs)  2'341 µs  3'341 µs  3'534 µs
Web Assembly: Latest     2'868 µs/iter   (2'753 µs … 4'308 µs)  2'850 µs  3'614 µs  4'308 µs
Web Assembly: Current    2'853 µs/iter   (2'750 µs … 3'597 µs)  2'848 µs  3'444 µs  3'597 µs
Node API: Current        1'421 µs/iter   (1'338 µs … 1'924 µs)  1'424 µs  1'760 µs  1'924 µs

summary for movies.findMany({ where: {...}, take: 2000 })
  Web Assembly: Current
   2.01x slower than Node API: Current
   1.22x slower than Web Assembly: Baseline
   1.01x faster than Web Assembly: Latest

• movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     575 ms/iter       (570 ms … 579 ms)    578 ms    579 ms    579 ms
Web Assembly: Latest       749 ms/iter       (747 ms … 754 ms)    752 ms    754 ms    754 ms
Web Assembly: Current      759 ms/iter       (750 ms … 772 ms)    769 ms    772 ms    772 ms
Node API: Current          467 ms/iter       (459 ms … 477 ms)    477 ms    477 ms    477 ms

summary for movies.findMany({ include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.63x slower than Node API: Current
   1.32x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline  79'980 µs/iter (79'437 µs … 80'432 µs) 80'229 µs 80'432 µs 80'432 µs
Web Assembly: Latest       106 ms/iter       (105 ms … 107 ms)    106 ms    107 ms    107 ms
Web Assembly: Current      107 ms/iter       (106 ms … 108 ms)    107 ms    108 ms    108 ms
Node API: Current       62'716 µs/iter (61'896 µs … 63'929 µs) 63'594 µs 63'929 µs 63'929 µs

summary for movies.findMany({ where: {...}, include: { cast: true } take: 2000 }) (m2m)
  Web Assembly: Current
   1.7x slower than Node API: Current
   1.33x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'020 ms/iter   (1'006 ms … 1'048 ms)  1'035 ms  1'048 ms  1'048 ms
Web Assembly: Latest     1'251 ms/iter   (1'245 ms … 1'272 ms)  1'249 ms  1'272 ms  1'272 ms
Web Assembly: Current    1'255 ms/iter   (1'241 ms … 1'275 ms)  1'265 ms  1'275 ms  1'275 ms
Node API: Current          899 ms/iter       (851 ms … 929 ms)    923 ms    929 ms    929 ms

summary for movies.findMany({ take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.4x slower than Node API: Current
   1.23x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline     143 ms/iter       (142 ms … 146 ms)    144 ms    146 ms    146 ms
Web Assembly: Latest       177 ms/iter       (173 ms … 186 ms)    180 ms    186 ms    186 ms
Web Assembly: Current      180 ms/iter       (178 ms … 185 ms)    180 ms    185 ms    185 ms
Node API: Current          111 ms/iter       (107 ms … 115 ms)    114 ms    115 ms    115 ms

summary for movie.findMany({ where: { ... }, take: 2000, include: { cast: { include: { person: true } } } })
  Web Assembly: Current
   1.62x slower than Node API: Current
   1.26x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

• movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'039 µs/iter     (977 µs … 1'717 µs)  1'035 µs  1'489 µs  1'717 µs
Web Assembly: Latest     1'372 µs/iter   (1'290 µs … 2'185 µs)  1'361 µs  2'126 µs  2'185 µs
Web Assembly: Current    1'378 µs/iter   (1'296 µs … 2'156 µs)  1'375 µs  2'007 µs  2'156 µs
Node API: Current          797 µs/iter     (758 µs … 1'145 µs)    812 µs    872 µs  1'145 µs

summary for movie.findMany({ where: { reviews: { author: { ... } }, take: 100 }) (to-many -> to-one)
  Web Assembly: Current
   1.73x slower than Node API: Current
   1.33x slower than Web Assembly: Baseline
   1x faster than Web Assembly: Latest

• movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
-------------------------------------------------------------- -----------------------------
Web Assembly: Baseline   1'030 µs/iter     (980 µs … 1'745 µs)  1'027 µs  1'683 µs  1'745 µs
Web Assembly: Latest     1'358 µs/iter   (1'297 µs … 2'195 µs)  1'358 µs  1'833 µs  2'195 µs
Web Assembly: Current    1'374 µs/iter   (1'299 µs … 2'374 µs)  1'369 µs  2'100 µs  2'374 µs
Node API: Current          801 µs/iter     (763 µs … 1'160 µs)    814 µs    857 µs  1'160 µs

summary for movie.findMany({ where: { cast: { person: { ... } }, take: 100 }) (m2m -> to-one)
  Web Assembly: Current
   1.72x slower than Node API: Current
   1.33x slower than Web Assembly: Baseline
   1.01x slower than Web Assembly: Latest

After changes in 82cfce5

Copilot AI review requested due to automatic review settings July 23, 2026 13:32
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ aqrln
❌ Weakky
You have signed the CLA already but the status is still pending? Let us recheck it.

@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 4a311a26-6501-4417-a5bb-23d39ce2e3b1

📥 Commits

Reviewing files that changed from the base of the PR and between e922089 and 3da6a45.

📒 Files selected for processing (1)
  • quaint/src/connector/connection_info.rs

Summary by CodeRabbit

  • Bug Fixes
    • Increased the default SQLite parameter limit, allowing larger batches of values in queries when no custom batch size is configured.

Walkthrough

Updates SQLite’s default maximum bind parameter count from 999 to 32766 in SqlFamily::default_max_bind_values(). The new value is used when calculating max_bind_values() without a QUERY_BATCH_SIZE override.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: raising SQLite's allowed query parameters to 32766.
Description check ✅ Passed The description matches the change by explaining the SQLite bind limit increase and its performance impact.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/sqlite_max_bind_values
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch fix/sqlite_max_bind_values

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates Quaint’s SQLite bind-parameter limit to reflect modern SQLite defaults, enabling larger batched queries and reducing unnecessary query chunking in Prisma Engines’ SQL query building layer.

Changes:

  • Increase SQLite max_bind_values default from 999 to 32766 in SqlFamily::default_max_bind_values().

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 494 to 503
pub fn default_max_bind_values(&self) -> usize {
match self {
#[cfg(feature = "postgresql")]
SqlFamily::Postgres => 32766,
#[cfg(feature = "mysql")]
SqlFamily::Mysql => 65535,
#[cfg(feature = "sqlite")]
SqlFamily::Sqlite => 999,
SqlFamily::Sqlite => 32766,
#[cfg(feature = "mssql")]
SqlFamily::Mssql => 2098,
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants