Added assertions for Arista devices to test_warmboot_data_consistency checker test#25489
Open
Ryangwaite wants to merge 1 commit into
Open
Conversation
Signed-off-by: Ryan Garthwaite <ryangwaite@gmail.com>
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
This PR has backport request for branch(es): 202605. ---Powered by SONiC BuildBot
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
Summary:
Fixes #21226
MSFT ADO: 38503975
Added assertions and an assertion mask framework for the warmboot data consistency test to validate Arista devices automatically. This enables detection of unexpected differences during warm-vs-cold boot comparisons.
Note the assertion mask is currently set to cold 202511.29 vs 202505.30--wb-->202511.29 Arista mask. Any failures on 202605 will be handled in follow-up PRs.
For platforms without an assertion mask, those tests continue to pass and require manual review of differences to assess if there is an issue.
Type of change
Back port request
Approach
What is the motivation for this PR?
Without this change the test doesn't have any assertions and requires manual effort to look through the diff which is time consuming. With this change, now for the 3 Arista platforms, their changes are assessed against a set of vetted and known diffs. If a new diff is found the test will fail and only at that point will the manual effort be required.
How did you do it?
warm_vs_cold_assertion_mask.pymodule that provides a declarative framework for masking expected post-prune diffs on a per-platform, per-upgrade-path basisdb_comparison.pywith expanded comparison logic and utilitiessnapshot_warm_vs_cold_boot_helpers.pyto support the new assertion frameworkwarm_vs_cold.pywith assertion mask application logictest_warmboot_data_consistency.pyto apply the assertion mask and assert no unmasked regressions remaindut_utils.pyto support the test improvementsHow did you verify/test it?
Tested cold 202511.29 vs 202505.30--wb-->202511.29 on Arista:
Any platform specific information?
This lays the assertion framework but only enables it for the following platforms:
Other platforms continue to have their current functionality
Supported testbed topology if it's a new test case?
Still T0*
Documentation