Search before asking
Description
Currently, BanyanDB may face limitations when multiple tags share the same name but utilize different data types (e.g., int vs str). That's due to the schema change.
This proposal suggests a new solution to allow tags with the same name and different types to coexist within the same part/segment by leveraging type-specific file naming conventions and refined query/merge logic.
This is a clear and well-defined proposal for managing tag name collisions across different types. Below is a draft for the GitHub issue, structured to meet the standards of the Apache SkyWalking repository.
Proposed Logic
1. Storage Structure (File-based)
Tags should be persisted in files that include both the tag name and the tag type in the filename to prevent overwriting at the filesystem level. Only add the tag type part when there is more than 1 tag type in the part.
- Format:
{tag_name}.{tag_type}.tf
- Example: *
state.int.tf (Integer type)
state.str.tf (String type)
2. Query Process
The query engine should no longer rely solely on the tag name. Instead, it must use a composite identifier:
- Input:
tag_name AND tag_type. They are defined in the projection.
- Selection: The query process selects the specific
.tf file that matches both the name and the requested type from the projection.
3. Merge & TTL Process
To maintain data consistency and handle updates:
- Merge: During the merging phase, only tags sharing both the same name and same type are merged together.
- TTL/Cleanup: When a tag's type is changed or updated, the previous version (with the old type) will be naturally phased out by the standard TTL (Time To Live) process, ensuring that orphaned type files do not persist indefinitely.
Use case
No response
Related issues
No response
Are you willing to submit a pull request to implement this on your own?
Code of Conduct
Search before asking
Description
Currently, BanyanDB may face limitations when multiple tags share the same name but utilize different data types (e.g., int vs str). That's due to the schema change.
This proposal suggests a new solution to allow tags with the same name and different types to coexist within the same part/segment by leveraging type-specific file naming conventions and refined query/merge logic.
This is a clear and well-defined proposal for managing tag name collisions across different types. Below is a draft for the GitHub issue, structured to meet the standards of the Apache SkyWalking repository.
Proposed Logic
1. Storage Structure (File-based)
Tags should be persisted in files that include both the tag name and the tag type in the filename to prevent overwriting at the filesystem level. Only add the tag type part when there is more than 1 tag type in the part.
{tag_name}.{tag_type}.tfstate.int.tf(Integer type)state.str.tf(String type)2. Query Process
The query engine should no longer rely solely on the tag name. Instead, it must use a composite identifier:
tag_nameANDtag_type. They are defined in the projection..tffile that matches both the name and the requested type from the projection.3. Merge & TTL Process
To maintain data consistency and handle updates:
Use case
No response
Related issues
No response
Are you willing to submit a pull request to implement this on your own?
Code of Conduct