Resource Trie Implementation - #133
Conversation
88503c6 to
afcf60c
Compare
There was a problem hiding this comment.
Code added here to track the amount of heap consumed in bm_malloc calls. Used in unit tests to validate memory was cleaned up with each test.
| (const uint8_t *)data, service_handler, 1000), | ||
| false); | ||
| free(data); | ||
| bm_free(data); |
There was a problem hiding this comment.
These calls had to be updated to support the new wrappers in bm_os_stub.c.
|
Note: In the mote test (adding and performing actions on 128 topics), there are a lot of pass-through nodes created due many splits that are occurring. The average alloc size is around 109 bytes because:
Let's say the average node is 50 bytes in total. This would mean that 2.18 trie elements are created for every resource due to the creation of pass-through elements! |
| ret = malloc(size); | ||
| ret = malloc(sizeof(MEM_ALLOC) + size); | ||
| size_t *alloc_size = (size_t *)ret; | ||
| *alloc_size = size; | ||
| MEM_ALLOC += size; | ||
| ret += sizeof(MEM_ALLOC); |
| // Test match before any add | ||
| err = resource_trie_match(&ROOT, topic); | ||
| EXPECT_EQ(err, BmOK); | ||
| EXPECT_EQ(ROOT.result.count, 0); |
There was a problem hiding this comment.
just getting going on reviewing the code, so not super familiar yet with how resource_trie_match is used yet, but my initial reaction to this is that resource_trie_match should not return BmOK when there is nothing to match against. I'll keep an eye out for how resource_trie_match is used and if this does make sense, but I thought I'd ask why is it that way?
There was a problem hiding this comment.
It certainly does not have to be that way! My thought process was: "We have successfully been able to traverse the trie without any errors and therefore do not need to return and error". Buttt, I can see how this could create some confusion, for example resource_trie_remove returns BmENODATA if it cannot delete the desired element, so maybe the better solution here is:
BmErr resource_trie_match(ResourceTrieRoot *root, const char *topic) {
...
if (!root->result.count) {
return BmENODATA;
}
return BmOK;
}Thoughts?
| // Strip concrete nodes of wildcard interests | ||
| bool is_wildcard = topic_has_wildcard(topic); | ||
| if (is_wildcard) { | ||
| resource_trie_match(root, topic); |
There was a problem hiding this comment.
Ahh I see, in relation to my comment in the unit tests, it looks like we don't actually check for the return value here. Is that something we should do? If not, would that warrant removing the return value from the function then?
There was a problem hiding this comment.
we definitely use the return value of resource_trie_match in the unit tests enough to not warrant removing the return value. so just wondering if we should also be checking here. If not maybe we could add a (void) to indicate we are purposefully ignoring
topic_len parameter to all API.
2 participants
Resource Trie
A compressed trie (radix tree) for topic-based pub/sub routing. Topics like
/sensor/temperature/raware split into segments and stored in a tree structure whereshared prefixes share elements, minimizing memory on constrained devices.
Stored topics are evaluated as concrete (no wildcard patterns) or wildcard (containing either
*or?).Node Structure
Each element in the trie is a
ResourceTrieElement:Children are a singly-linked list:
parent->childrenpoints to the first child,each child's
->siblingpoints to the next.Passthrough elements have
resource_id == invalid_resource_id(0x1FFFFF). Theyexist only as structural connectors in the trie — no resource has expressed interest at
that path. They are created by splits (when adding a node) and cleaned up by
compression (when removing nodes).
Operations
Add (
resource_trie_add)Descends the trie via
exact_match, consuming topic segments. Three outcomes:1. Topic already exists — update the existing node's resource fields.
2. Topic partially matches an existing segment — split the node, then continue.
3. No match at current level — allocate a new leaf and attach it.
When a wildcard is added to the trie, the
is_wildcardstruct member flag is set.After a new element is added to the trie, the following is performed based on the type of
element added:
concrete: The trie is searched for all matching wildcard resources,for every match the following is updated:
wildcard_port_maskis OR'd with the wildcard resourcewildcard_interestis OR'd with the wildcard resourcewildcard: The trie is searched for all matching concrete resources,for every match the following is updated:
wildcard_port_maskis OR'd with the wildcard resourcewildcard_interestis OR'd with the wildcard resourceThis will be used in resource based routing to correlate concrete topics with wildcard topics
to ensure messages are routed properly throughout the network.
Example: Building a trie
The second add causes a split:
"sensor/temp"shares prefix"sensor"with"sensor/pressure", so the node splits into a passthrough (pt)"sensor"with twochildren.
When the last element is added
"/sensor/*", it matches every previously added resource.Meaning, those resources'
wildcard_port_maskandwildcard_interestwill be updated with"/sensor/*"'sport_mask,0x0010, andlocal_interest,true).Match (
resource_trie_match)Uses
match_wildcardto perform a full trie traversal with an iterativein-order walk (Depth First Search). Reconstructs each node's full topic path in
root->match_strand compares against the query topic. Supports both concrete and wildcard matching.
Results are pointers to individual elements stored in
root->result.matches[](up to 64).In order to descend the trie, a stack data structure is used,
root->stack.The stack is large enough to descend into the depth of the trie if every topic
segment was a single character (max of 255 characters or 127 possible trie elements).
An example of how this works in practice is as follows:
Remove (
resource_trie_remove)Removal has three phases, with an additional fourth phase when the removed topic
is a wildcard:
Phase 1: Find and track ancestry
exact_matchdescends to the target node. Thetrack_ancestrycallback pusheseach ancestor onto
root->stackduring descent:Phase 2: Remove the node (
remove_child)Three cases depending on the target node:
In Case 1,
check_and_compressattempts to merge the invalidated node with itschild if it has exactly one and parent if the parent is a pass-through node.
Phase 3: Ancestor compression
Walk the ancestry stack bottom-up, calling
check_and_compressat each level.This cleans up passthrough chains created by the removal.
In the removal case of
/a/b/c/dthe new trie will look like:Phase 4: Wildcard cross-reference cleanup (wildcard topics only)
When the removed topic is a wildcard, concrete topics that were matched by it
still carry stale
wildcard_port_maskandwildcard_interestbits from thenow-deleted wildcard. This phase is the inverse of the cross-reference update
performed during add.
For each concrete topic that matches the removed wildcard pattern:
Strip the removed wildcard's bits:
wildcard_port_mask &= ~removed_port_maskwildcard_interest &= ~removed_local_interestRe-aggregate from any remaining wildcards — other wildcards may share the
same bits, so the concrete topic is re-matched against the trie to restore
any bits that are still valid.
Compression (
check_and_compress)Compression merges a passthrough parent with its single child, eliminating one
level of indirection:
The operation:
parent_segment + "/" + child_segment*parent = *child)Compression guards (any true = skip compression operation):
!childchild->siblingparent == &root->elementparent->resource_id != invalidCompression Example
Remove "/a/b/c":
Remove "/a/x":
Remove "/a/b/d":
Split (
split_element)When a new topic shares a partial prefix with an existing compressed segment,
the segment is split at the divergence point.
Split Example
The original node keeps its resource and becomes a child. The new passthrough
takes over the shared prefix. This is the only way passthrough nodes are created.
Key Invariants
Passthrough nodes always have children. Passthroughs are created by
split_element(which produces 2+ children). Compression fires whenever apassthrough reaches exactly 1 child, absorbing it. This prevents passthroughs
from ever reaching 0 children and leaking memory.
Compression never destroys valid resources. The
parent_has_resourceguard ensures only passthrough nodes are overwritten during compression. A
node with a real subscription resource is never compressed away.
Ancestor cleanup is bounded. The ancestry stack is populated during
exact_matchdescent and has at mostresource_trie_max_depthentries(BM_TOPIC_MAX_LEN / 2 = 127). The compression loop breaks on the first
non-compressible ancestor.
API
Memory Model
All allocations go through
bm_malloc/bm_free.Each element requires two allocations: the
ResourceTrieElementstruct and itssegment string upon being created.
ResourceTrieRootis stack-allocated by the caller and containsthe root sentinel element, the traversal stack, match results buffer, and match
string workspace.
Mote Testing
I created a test for the mote that looks at the latency and memory usage for the resource trie implementation.
The diff for this can be applied to
bm_protocolhere:resource_trie_mote_app_diff.txt
And then this can be built with:
This test implements a resource trie with 128 unique topics, 96 concrete topics and 32 wildcard topics. The test performs the following actions:
The output for this test can be seen below: