Skip to content

Support multi-attribute GSI partition and sort keys#206

Open
cluebbehusen wants to merge 4 commits into
HENNGE:masterfrom
cluebbehusen:multi-attribute-gsi-keys
Open

Support multi-attribute GSI partition and sort keys#206
cluebbehusen wants to merge 4 commits into
HENNGE:masterfrom
cluebbehusen:multi-attribute-gsi-keys

Conversation

@cluebbehusen

Copy link
Copy Markdown

Closes #202

DynamoDB added support for multi-attribute composite keys on GSIs late last year. This PR adds support for that feature.

Docs: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GSI.DesignPattern.MultiAttributeKeys.html

KeySchema now accepts tuples of KeySpec for hash_key and range_key, with validation enforcing DynamoDB's 1-4 attribute limit per key type. Single KeySpec values still work as before.

MultiHashKey is a new expression class for querying multi-attribute partition keys:

# Table creation
gsi = GlobalSecondaryIndex(
    name="tenant-region-index",
    schema=KeySchema(
        hash_key=(KeySpec("tenant", KeyType.string), KeySpec("region", KeyType.string)),
        range_key=(KeySpec("date", KeyType.string), KeySpec("seq", KeyType.number)),
    ),
    projection=Projection(ProjectionType.all),
    throughput=None,
)

# Querying
async for item in client.query(
    "my-table",
    MultiHashKey(("tenant", "acme"), ("region", "us-east"))
    & RangeKey("date").equals("2025-01-01")
    & RangeKey("seq").gt(0),
    index="tenant-region-index",
):
    pass

Also fixes HashAndRangeKeyCondition missing __and__, which meant chaining multiple RangeKey conditions required explicit parentheses.

@dimaqq dimaqq left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not the maintainer, but used to work on this codebase.

I think a change like this should come with a new or updated integration test.

Add fixture and tests that exercise multi-attribute composite partition
and sort keys against DynamoDB Local (skipped on unsupported backends).
@cluebbehusen

Copy link
Copy Markdown
Author

I'm not the maintainer, but used to work on this codebase.

I think a change like this should come with a new or updated integration test.

Thanks, that's a good point! I've added integration tests in the latest commit. They cover various cases that DynamoDB supports.

Unfortunately, not all integration testing backends support this new feature yet. DynamoDB local does, and I validated the tests against that backend. Unfortunately, Scylla and dynalite do not, so I set up the tests to skip on those backends.

FWIW, moto supports this new feature, but I didn't see that as an existing backend. I'd be happy to add that though if desired.

@ojii

ojii commented Mar 9, 2026

Copy link
Copy Markdown
Contributor

Unfortunately, not all integration testing backends support this new feature yet.

you can use the flavor fixture to skip tests on some implementations.

@cluebbehusen

Copy link
Copy Markdown
Author

Unfortunately, not all integration testing backends support this new feature yet.

you can use the flavor fixture to skip tests on some implementations.

@ojii, I did set up the tests to skip on unsupported backends.

One nuance is that I had to base it off of the backend name rather than flavor. I did this because there is no flavor for DynamoDB local or localstack; they are both other. DynamoDB local works, but localstack does not, so I could not control skipping based on other.

I would be happy to add more granular flavors, but that felt like scope creep on this PR. Let me know if you would prefer expanding flavor though, and I'd be happy to make that change.

@ojii

ojii commented Mar 9, 2026

Copy link
Copy Markdown
Contributor

I would be happy to add more granular flavors, but that felt like scope creep on this PR. Let me know if you would prefer expanding flavor though, and I'd be happy to make that change.

expanding makes sense. basically I only added different flavors if they were needed because the implementations differed (I think currently we don't use it since all implementations are similar enough for the tests)

Use flavor enum instead of implementation_name for skipping
multi-attribute GSI tests on unsupported backends. Remove the now-unused
implementation_name fixture.
@cluebbehusen

Copy link
Copy Markdown
Author

I would be happy to add more granular flavors, but that felt like scope creep on this PR. Let me know if you would prefer expanding flavor though, and I'd be happy to make that change.

expanding makes sense. basically I only added different flavors if they were needed because the implementations differed (I think currently we don't use it since all implementations are similar enough for the tests)

Perfect! I expanded flavor and updated the tests to use that. I removed the implementation_name fixture I had added since it's now unused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support DynamoDB's new multi-attribute GSI partition and sort keys

3 participants