Skip to content

feat(qis): lower shared pointer cells through runtime hooks - #2052

Draft
doug-q wants to merge 11 commits into
mainfrom
doug/pointer-codegen
Draft

doug-q wants to merge 11 commits into
mainfrom
doug/pointer-codegen

Conversation

@doug-q

@doug-q doug-q commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Pointer HUGRs currently have no QIS codegen registration. Register HUGR's pointer lowering for Sol and Helios with a complete QisPtrCodegen backend whose runtime owns storage, reference counts, synchronization and final payload recovery. HUGR's generic operation lowering handles typed values and control flow; this backend adds no second counter to the payload.

The backend implements all six required hooks together: New transfers an initialized value into a cell with one owner; Dup retains one owner; Free consumes one owner and returns a final-owner flag, writing the moved value only when final. Lifecycle hooks are entered unlocked. Read/Write/Swap/Map use locked value-only projection; Map keeps the lock across its callback. Eq compares allocation identity, returns both original linear handles in input order and emits no hooks or payload/refcount access.

Proposed runtime ABI (test-only implementations exist in this draft):

  • ___ptr_create(u64 size, u64 alignment, ptr source) -> ptr: transfer initial value bytes, including potentially uninitialized padding, into aligned storage; establish one owner and synchronization resources; return a non-null handle or terminate.
  • ___ptr_inc_refcount(ptr handle, i64 delta, ptr destination) -> bool: Dup uses +1/null; Free uses -1/non-null typed destination. A final release moves the payload and retires the cell internally. Otherwise destination is untouched. Extraction is entered without the caller holding the payload lock, and no access/unlock of the consumed handle follows. Null negative adjustment would discard without a destructor; generated Free always recovers its linear value.
  • ___ptr_get_ptr(ptr) -> ptr and ___ptr_lock/unlock(ptr) -> void: project only the value while locked, with mutual exclusion and acquire/release synchronization.

These are proposed exports, not available production runtime support. Runtime implementation/agreement, failure/cancellation handling and calling-convention compatibility remain merge prerequisites. There is no unsynchronized fallback. Same-cell re-entry during Map and callbacks that do not return normally are unsupported. Thread returned handles or use order edges to constrain operations on duplicated handles. No runtime-specific dependencies or downstream runtime integration are included.

Depends on HUGR #3282 at 74e4fc0df557f30e746d453473890d5dd044bff4 and stacked #3285 at fc654a9ad354f7235ad2469b6b3d10a43baf94d1. Direct Rust and Python sources use the stacked revision, with Cargo.lock and uv.lock retained; only the required HUGR source revision changes. The complete HUGR default backend still explicitly implements libc allocation, LLVM-managed counting and no-op locks; the QIS runtime backend supplies its own coherent ownership strategy.

Instrumented JIT coverage uses runtime metadata separate from value-only storage and checks payload size, signed owner adjustments, no lifecycle lock/projection, final recovery/destruction exactly once, no access/unlock after release, nested linear values, and concurrent Map. Equality tests cover aliases, distinct equal payloads and ordered distinct handles. Compiler tests exercise the normal Sol/Helios preparation pipeline and confirm equality emits no runtime calls. These test shims establish no production execution support.

The required HUGR revision also deprecates Graphviz emission; a narrow expectation preserves tket's existing dot_string API. The semver workflow compares an isolated checkout of the actual tested merge against its first parent, avoiding stale event SHAs and nested-checkout ambiguity.

Validation against the pinned published revision: 285 tests passed (17 compiler, 250 qsystem unit and 18 integration; one existing ignored test), all-target/all-feature Clippy with -D warnings, uv lock consistency and formatting passed. Builds used the pinned devenv environment, two Cargo jobs and an isolated RAM-backed target because disk space was low; retained artifacts and unrelated package versions were preserved. Two additional Python-feature bitcode-serialization tests fail locally with the same invalid-bitcode diagnostic in the retained pre-change binary; this baseline limitation is unrelated to pointer ownership.

Base synchronization: merge 711882f126dd093fc9ac76fc267742d3ecacbf8c brings verified main 19da7017f1436a171d9f86cc6e1da8398ca4bfdd into this draft and resolves the rstest lockfile conflict while retaining HUGR fc654a9a in Rust and Python. Main already includes the separately reviewable PyO3 Clippy1.99 fix #2056; no local lint suppression or type/API change was added.

After synchronization, pinned locked validation passes all 285 pointer/compiler tests, all-target/all-feature Clippy for tket-py, tket-qsystem and the QIS compiler with warnings denied, formatting and uv lock --check. The separately reviewable test-only correction 3226e48c replaces deprecated atomic fetch_update with a checked compare-exchange loop, retaining the same memory ordering and MSRV compatibility. Six focused execution tests and warning-denying qsystem Clippy passed after that change.

PR-triggered CI at final head 3226e48c5fb50a63120b6b1b959ece804ca3f1a6 completed successfully: full CI, semver and docs. Rust1.99 lint, QIS Rust/Python, stable/beta/nightly/MSRV tests, Python coverage, extension checks and benchmarks pass. Semver performed real package comparisons against main 19da7017 (NEW_PACKAGE=false). The local public-bitcode baseline limitation documented above remains separate.

@codecov

codecov Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.51451% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.34%. Comparing base (19da701) to head (3226e48).

Files with missing lines Patch % Lines
tket/tket-qsystem/src/llvm/ptr.rs 89.25% 3 Missing and 10 partials ⚠️
tket/tket-qsystem/src/llvm/ptr/test/runtime.rs 95.50% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2052      +/-   ##
==========================================
+ Coverage   83.23%   83.34%   +0.10%     
==========================================
  Files         244      247       +3     
  Lines       39666    40045     +379     
  Branches    37965    38344     +379     
==========================================
+ Hits        33015    33374     +359     
- Misses       5367     5377      +10     
- Partials     1284     1294      +10     
Flag Coverage Δ
python 92.39% <ø> (ø)
qis-compiler 74.37% <ø> (ø)
rust 82.94% <95.51%> (+0.11%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant