Skip to content

Added assertions for Arista devices to test_warmboot_data_consistency checker test#25489

Open
Ryangwaite wants to merge 1 commit into
sonic-net:masterfrom
Ryangwaite:add-assertions-to-wb-data-consistency-test
Open

Added assertions for Arista devices to test_warmboot_data_consistency checker test#25489
Ryangwaite wants to merge 1 commit into
sonic-net:masterfrom
Ryangwaite:add-assertions-to-wb-data-consistency-test

Conversation

@Ryangwaite

@Ryangwaite Ryangwaite commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Description of PR

Summary:
Fixes #21226
MSFT ADO: 38503975

Added assertions and an assertion mask framework for the warmboot data consistency test to validate Arista devices automatically. This enables detection of unexpected differences during warm-vs-cold boot comparisons.

Note the assertion mask is currently set to cold 202511.29 vs 202505.30--wb-->202511.29 Arista mask. Any failures on 202605 will be handled in follow-up PRs.

For platforms without an assertion mask, those tests continue to pass and require manual review of differences to assess if there is an issue.

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • New Test case
    • Skipped for non-supported platforms
  • Test case improvement

Back port request

  • 202311
  • 202405
  • 202411
  • 202505
  • 202511
  • 202512
  • 202605

Approach

What is the motivation for this PR?

Without this change the test doesn't have any assertions and requires manual effort to look through the diff which is time consuming. With this change, now for the 3 Arista platforms, their changes are assessed against a set of vetted and known diffs. If a new diff is found the test will fail and only at that point will the manual effort be required.

How did you do it?

  1. Created a new warm_vs_cold_assertion_mask.py module that provides a declarative framework for masking expected post-prune diffs on a per-platform, per-upgrade-path basis
  2. Enhanced db_comparison.py with expanded comparison logic and utilities
  3. Extended snapshot_warm_vs_cold_boot_helpers.py to support the new assertion framework
  4. Updated warm_vs_cold.py with assertion mask application logic
  5. Modified test_warmboot_data_consistency.py to apply the assertion mask and assert no unmasked regressions remain
  6. Added helper utilities in dut_utils.py to support the test improvements

How did you verify/test it?

Tested cold 202511.29 vs 202505.30--wb-->202511.29 on Arista:

  • Arista-7050CX3-32S-C32
  • Arista-7260CX3-D108C8
  • Arista-7060CX-32S-D48C8

Any platform specific information?

This lays the assertion framework but only enables it for the following platforms:

  • x86_64-arista_7260
  • x86_64-arista_7060
  • x86_64-arista_7050cx3

Other platforms continue to have their current functionality

Supported testbed topology if it's a new test case?

Still T0*

Documentation

Signed-off-by: Ryan Garthwaite <ryangwaite@gmail.com>
@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld

Copy link
Copy Markdown
Collaborator

This PR has backport request for branch(es): 202605.
Added label(s) for branch(es) 202605.

---Powered by SONiC BuildBot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test Gap: Add assertions to warm vs cold boot snapshot comparison test

2 participants