Skip to content

Add variable shape tensor#19

Merged
JohanMabille merged 7 commits into
sparrow-org:mainfrom
Alex-PLACET:add_variable_shape_tensor
Jan 6, 2026
Merged

Add variable shape tensor#19
JohanMabille merged 7 commits into
sparrow-org:mainfrom
Alex-PLACET:add_variable_shape_tensor

Conversation

@Alex-PLACET

Copy link
Copy Markdown
Collaborator

No description provided.

@Alex-PLACET
Alex-PLACET marked this pull request as ready for review December 24, 2025 10:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements the Variable Shape Tensor Array extension following the Apache Arrow canonical extension specification. The implementation allows efficient storage and transfer of multi-dimensional tensors with varying shapes.

Key changes include:

  • Addition of variable_shape_tensor_array class with metadata serialization/deserialization
  • Implementation of Arrow extension metadata handling and validation
  • Migration from nlohmann_json to simdjson for improved JSON parsing performance

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/test_variable_shape_tensor.cpp Comprehensive test suite for metadata validation, JSON serialization, and round-trip operations
tests/CMakeLists.txt Registers new test file in the build system
src/variable_shape_tensor.cpp Core implementation of metadata, serialization, and array construction
include/sparrow_extensions/variable_shape_tensor.hpp Header defining public API and template implementations
include/sparrow_extensions.hpp Adds variable_shape_tensor header to main include
environment-dev.yml Updates dependency from nlohmann_json to simdjson
docs/source/variable_shape_tensor_array.md Comprehensive documentation with usage examples and API reference
docs/source/main_page.md Adds documentation link for fixed_shape_tensor_array
cmake/external_dependencies.cmake Adds simdjson dependency configuration
CMakeLists.txt Registers variable_shape_tensor.cpp in build sources

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/source/main_page.md Outdated
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Comment thread include/sparrow_extensions/variable_shape_tensor.hpp Outdated
Comment thread include/sparrow_extensions/variable_shape_tensor.hpp
Comment thread include/sparrow_extensions/variable_shape_tensor.hpp Outdated
Comment thread src/variable_shape_tensor.cpp Outdated
@JohanMabille
JohanMabille merged commit 9172562 into sparrow-org:main Jan 6, 2026
42 checks passed
@Alex-PLACET
Alex-PLACET deleted the add_variable_shape_tensor branch January 6, 2026 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants