Skip to content

chore: pin Arrow to 58.3 to match DataFusion v54#78

Merged
lukekim merged 2 commits into
trunkfrom
chore/arrow-58.3-datafusion-54
Jul 13, 2026
Merged

chore: pin Arrow to 58.3 to match DataFusion v54#78
lukekim merged 2 commits into
trunkfrom
chore/arrow-58.3-datafusion-54

Conversation

@lukekim

@lukekim lukekim commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Aligns the SDK's Arrow dependency with the exact version DataFusion v54 ships.

DataFusion v54.0.0 depends on arrow ^58.3.0 (confirmed against the crates.io API and the apache/datafusion 54.0.0 tag — arrow, arrow-flight, arrow-schema, parquet all 58.3.0).

The SDK was already on the Arrow 58 major (from #75), which is semver-compatible. This raises the floor to 58.3 so the SDK pins to the exact minor DataFusion v54 uses — same <59 ceiling — and any downstream project combining this SDK with DataFusion 54 resolves to a single Arrow 58.3.x.

-arrow        = { version = "58",   features = ["prettyprint"] }
-arrow-flight = { version = "58",   features = ["flight-sql-experimental"] }
-arrow-json   = "58"
+arrow        = { version = "58.3", features = ["prettyprint"] }
+arrow-flight = { version = "58.3", features = ["flight-sql-experimental"] }
+arrow-json   = "58.3"

Follows the repo's existing convention (#67, "Explicitly upgrade to Arrow 57.2").

Verification

  • arrow / arrow-flight / arrow-json resolve to 58.3.0.
  • cargo check --all-targets — compiles clean (only a pre-existing dead-code warning, unrelated).
  • cargo test --lib178 passed, 0 failed, including the Arrow schema/RecordBatch tests.
  • tonic / prost already at 0.14 (what arrow-flight 58 / DataFusion v54 require) — no change needed.

The live-runtime integration tests in tests/client_test.rs are unaffected by this change (they require a running Spice runtime).

🤖 Generated with Claude Code

DataFusion v54 depends on arrow ^58.3.0 (arrow, arrow-flight, arrow-schema,
parquet). The SDK was already on the 58 major (#75); this raises the floor to
the exact minor DataFusion v54 ships, so downstream projects that combine this
SDK with DataFusion 54 resolve to a single Arrow 58.3.x.

- arrow / arrow-flight / arrow-json: "58" -> "58.3" (same <59 ceiling)
- Resolves to arrow 58.3.0; builds clean; 178 offline unit tests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@lukekim lukekim self-assigned this Jul 13, 2026
@lukekim lukekim requested a review from Copilot July 13, 2026 03:45
@lukekim lukekim added this to the v4.0.0 milestone Jul 13, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Aligns this Rust SDK’s Apache Arrow dependency versions with the minor release used by DataFusion v54, aiming to reduce downstream version resolution conflicts when both are used together.

Changes:

  • Bumps arrow, arrow-flight, and arrow-json dependency requirements from "58" to "58.3" in Cargo.toml.

Comment thread Cargo.toml
The macOS build jobs fail intermittently in the "Install Spice" step, not in
Rust: `cargo clippy` compiles clean on 1.93.1 in the same job. The Homebrew
formula's post-install ("Upgrading Spice Runtime") flakes and makes
`brew install` exit non-zero even though the `spice` CLI installs fine. This
is pre-existing on trunk (macos-14 fails there too), independent of the Arrow
bump, and the failing macOS cell varies run to run.

- macOS Spice install: tolerate the flaky brew post-install and fetch the
  runtime explicitly with retries so a transient hiccup can't fail the job.
- Replace the archived actions-rs/toolchain (deprecated, Node 20) with the
  maintained, SHA-pinned dtolnay/rust-toolchain. The concrete toolchain stays
  governed by rust-toolchain.toml (channel = "1.93.1").

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@lukekim lukekim enabled auto-merge (squash) July 13, 2026 04:58
@lukekim lukekim merged commit 3070059 into trunk Jul 13, 2026
16 checks passed
@lukekim lukekim deleted the chore/arrow-58.3-datafusion-54 branch July 13, 2026 05:27
lukekim added a commit that referenced this pull request Jul 13, 2026
`curl https://install.spiceai.org | /bin/bash` has no `-f` flag, so when the
proxied GitHub script returns an HTTP 429 rate-limit page, that page is piped
into bash and executed (syntax error → exit 2) — an intermittent flake seen on
PR CI. Add `-f` + `pipefail` so a bad HTTP response is a real curl failure, and
retry the installer and `spice install` (mirrors the resilient macOS step from
#72/#78).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
lukekim added a commit that referenced this pull request Jul 13, 2026
…install) (#79)

* chore: cargo fmt --all + add rustfmt CI gate

Trunk had accumulated rustfmt drift — CI never ran `cargo fmt --check` and
there is no rustfmt.toml, so pre-existing code had multi-line attributes,
unwrapped imports, and inconsistent call wrapping.

- Run `cargo fmt --all` across the workspace (src/params.rs, src/client.rs).
- Add a `rustfmt` job to build.yml (`cargo fmt --all --check`) so formatting
  stays clean going forward.

Pure formatting — no behavioral change; all lib tests and doctests pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* ci: make Linux Spice install resilient to GitHub 429 rate limits

`curl https://install.spiceai.org | /bin/bash` has no `-f` flag, so when the
proxied GitHub script returns an HTTP 429 rate-limit page, that page is piped
into bash and executed (syntax error → exit 2) — an intermittent flake seen on
PR CI. Add `-f` + `pipefail` so a bad HTTP response is a real curl failure, and
retry the installer and `spice install` (mirrors the resilient macOS step from
#72/#78).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants