Implement CAP-86 - add host functions for efficient 'sparse' map creation/reads.#1713
Open
dmkozh wants to merge 2 commits into
Open
Implement CAP-86 - add host functions for efficient 'sparse' map creation/reads.#1713dmkozh wants to merge 2 commits into
dmkozh wants to merge 2 commits into
Conversation
…tion/reads. This adds `sparse_map_new_from_linear_memory` and `sparse_map_unpack_to_linear_memory` host functions, as well as the native host implementation for their slice-based counterparts. While working on this, I've noticed that the existing host functions for map creation/packing lack test coverage, so this adds behavior tests both for the old and new functions. The code has also been refactored to minimize duplication between the old/new versions and between the slice/lineary memory versions.
Contributor
There was a problem hiding this comment.
Pull request overview
Implements CAP-86 sparse map construction and unpacking across the common ABI, guest, and host layers, with expanded behavioral tests and updated resource observations.
Changes:
- Adds sparse map slice and linear-memory host functions.
- Refactors dense and sparse map handling into shared implementations.
- Adds map behavior tests and refreshes resource baselines.
Reviewed changes
Copilot reviewed 55 out of 97 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
soroban-simulation/src/test/simulation.rs |
Updates simulation resource baselines. |
soroban-env-host/src/testutils.rs |
Adds configurable map Wasm fixtures. |
soroban-env-host/src/test/stellar_asset_contract.rs |
Updates instruction baseline. |
soroban-env-host/src/test/map.rs |
Adds dense and sparse map tests. |
soroban-env-host/src/test/lifecycle.rs |
Updates instruction baseline. |
soroban-env-host/src/test/auth.rs |
Updates authorization baselines. |
soroban-env-host/src/host/mem_helper.rs |
Extracts slice scanning and symbol comparison. |
soroban-env-host/src/host/data_helper.rs |
Implements shared map operations. |
soroban-env-host/src/host.rs |
Routes map APIs through shared implementations. |
soroban-env-host/observations/test__stellar_asset_contract__test_cannot_burn_native.json |
Refreshes CPU observations. |
soroban-env-host/observations/test__stellar_asset_contract__test_auth_rejected_for_bad_signature_type.json |
Refreshes CPU observations. |
soroban-env-host/observations/test__stellar_asset_contract__cap_73__test_xlm_transfer_muxed_creates_account.json |
Refreshes CPU observations. |
soroban-env-host/observations/test__stellar_asset_contract__cap_73__test_xlm_transfer_fails_below_minimum.json |
Refreshes CPU observations. |
soroban-env-host/observations/test__stellar_asset_contract__cap_73__test_xlm_transfer_creates_account.json |
Refreshes CPU observations. |
soroban-env-host/observations/test__stellar_asset_contract__cap_73__test_trust_noop_if_trustline_exists.json |
Refreshes CPU observations. |
soroban-env-host/observations/test__stellar_asset_contract__cap_73__test_trust_noop_for_contract_address.json |
Refreshes CPU observations. |
soroban-env-host/observations/test__stellar_asset_contract__cap_73__test_trust_fails_for_issuer.json |
Refreshes CPU observations. |
soroban-env-host/observations/test__stellar_asset_contract__cap_73__test_trust_fails_account_missing.json |
Refreshes CPU observations. |
soroban-env-host/observations/test__map__sparse_map_unpack_to_slice.json |
Records sparse slice unpacking. |
soroban-env-host/observations/test__map__sparse_map_unpack_to_linear_memory_roundtrip.json |
Records sparse memory roundtrip. |
soroban-env-host/observations/test__map__sparse_map_new_from_slices.json |
Records sparse slice creation. |
soroban-env-host/observations/test__map__sparse_map_new_from_linear_memory_skips_void.json |
Records void filtering. |
soroban-env-host/observations/test__map__map_new_and_unpack__test_sparse_map_unpack_to_slice.json |
Records sparse unpack tests. |
soroban-env-host/observations/test__map__map_new_and_unpack__test_sparse_map_new_from_slices.json |
Records sparse creation tests. |
soroban-env-host/observations/test__map__map_new_and_unpack__test_map_unpack_to_slice.json |
Records dense unpack tests. |
soroban-env-host/observations/test__map__map_new_and_unpack__test_map_new_from_slices.json |
Records dense creation tests. |
soroban-env-host/observations/test__map__map_build_bad_element_integrity.json |
Updates integrity-error traces. |
soroban-env-host/observations/test__map__linear_memory_operations.json |
Updates memory-operation costs. |
soroban-env-host/observations/test__map__instantiate_oversized_map_from_linear_memory.json |
Updates oversized-map observations. |
soroban-env-host/observations/test__map__initialization_invalid.json |
Updates invalid-input traces. |
soroban-env-host/observations/test__linear_memory__test_calling_map_unpack_to_linear_memory_with_pos_out_of_range.json |
Updates bounds-error cost. |
soroban-env-host/observations/test__linear_memory__test_calling_map_unpack_to_linear_memory_with_length_too_long.json |
Updates bounds-error trace. |
soroban-env-host/observations/test__linear_memory__test_calling_map_unpack_to_linear_memory_with_good_val_data_ok.json |
Updates successful unpack cost. |
soroban-env-host/observations/test__linear_memory__test_calling_map_unpack_to_linear_memory_with_good_slices_ok.json |
Updates successful slice cost. |
soroban-env-host/observations/test__linear_memory__test_calling_map_unpack_to_linear_memory_with_bad_key_ptr.json |
Updates bad-pointer cost. |
soroban-env-host/observations/test__linear_memory__test_calling_map_unpack_to_linear_memory_with_bad_key_length.json |
Updates bad-length cost. |
soroban-env-host/observations/test__linear_memory__test_calling_map_new_from_linear_memory_with_invalid_vals.json |
Updates invalid-value cost. |
soroban-env-host/observations/test__dispatch__invalid_object_handle_sparse_map_unpack_to_linear_memory_arg_0.json |
Adds sparse dispatch validation. |
soroban-env-guest/src/guest.rs |
Adds guest sparse slice adapters. |
soroban-env-common/src/symbol.rs |
Adds identity Symbol conversion. |
soroban-env-common/src/env.rs |
Exposes native sparse map APIs. |
soroban-env-common/env.json |
Defines protocol-28 host functions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
This adds
sparse_map_new_from_linear_memoryandsparse_map_unpack_to_linear_memoryhost functions, as well as the native host implementation for their slice-based counterparts.While working on this, I've noticed that the existing host functions for map creation/packing lack test coverage, so this adds behavior tests both for the old and new functions. The code has also been refactored to minimize duplication between the old/new versions and between the slice/linear memory versions.
Why
Implementation of CAP-86
Known limitations
N/A