Skip to content

Attempt to fix flaky graphql OrderStatusChangeDateTest.#40618

Open
hostep wants to merge 1 commit intomagento:2.4-developfrom
hostep:fix-flaky-order-status-change-date-test
Open

Attempt to fix flaky graphql OrderStatusChangeDateTest.#40618
hostep wants to merge 1 commit intomagento:2.4-developfrom
hostep:fix-flaky-order-status-change-date-test

Conversation

@hostep
Copy link
Copy Markdown
Contributor

@hostep hostep commented Mar 22, 2026

Description (*)

While working on another PR, noticed a failure in the WebAPI Tests suite: https://public-results-storage-prod.magento-testing-service.engineering/reports/magento/magento2/pull/40612/0094f73f811ad6da1dd84b165c063b6c/WebApi/console-error-logs.html

There was 1 failure:

  1. Magento\GraphQl\Sales\OrderStatusChangeDateTest::testOrderStatusChangeDateWithStatusChange
    Failed asserting that two strings are equal.
    --- Expected
    +++ Actual
    @@ @@
    -'2026-03-21 21:28:35'
    +'2026-03-21 21:28:36'

/var/www/html/dev/tests/api-functional/testsuite/Magento/GraphQl/Sales/OrderStatusChangeDateTest.php:95
/var/www/html/dev/tests/api-functional/testsuite/Magento/GraphQl/Sales/OrderStatusChangeDateTest.php:65

The assumption is that the Order object has an updatedAt timestamp in memory which differs from the database, as that timestamp is probably generated by MySQL itself and not in PHP.
When later asserting its value, the timestamp comes from the in memory object and not from the database, which can lead to millisecond differences and if the test is ran very close to the end of second, then maybe this flakyness can be explained.
I'm trying to fix it by loading the order from the database before fetching the updatedAt timestamp. Which should hopefully fix this problem. Let's see if this change keeps the test running smoothly🤞

Related Pull Requests

N/A

Fixed Issues (if relevant)

N/A

Manual testing scenarios (*)

  1. Run the test, and see if it keeps running successfully, hopefully with fewer flakyness
  2. Maybe we need to run this test like 100 times to see if never errors out anymore, but who has time for that...

Questions or comments

I didn't ran this test locally, let's see how it runs in this PR to see if it keeps working as expected.

Contribution checklist (*)

  • Pull request has a meaningful description of its purpose
  • All commits are accompanied by meaningful commit messages
  • All new or changed code is covered with unit/integration tests (if applicable)
  • README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
  • All automated tests passed successfully (all builds are green)

@m2-assistant
Copy link
Copy Markdown

m2-assistant bot commented Mar 22, 2026

Hi @hostep. Thank you for your contribution!
Here are some useful tips on how you can test your changes using Magento test environment.
❗ Automated tests can be triggered manually with an appropriate comment:

  • @magento run all tests - run or re-run all required tests against the PR changes
  • @magento run <test-build(s)> - run or re-run specific test build(s)
    For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names.

Allowed build names are:
  1. Database Compare
  2. Functional Tests CE
  3. Functional Tests EE
  4. Functional Tests B2B
  5. Integration Tests
  6. Magento Health Index
  7. Sample Data Tests CE
  8. Sample Data Tests EE
  9. Sample Data Tests B2B
  10. Static Tests
  11. Unit Tests
  12. WebAPI Tests
  13. Semantic Version Checker

You can find more information about the builds here
ℹ️ Run only required test builds during development. Run all test builds before sending your pull request for review.


For more details, review the Code Contributions documentation.
Join Magento Community Engineering Slack and ask your questions in #github channel.

@hostep
Copy link
Copy Markdown
Contributor Author

hostep commented Mar 22, 2026

@magento run all tests

@hostep
Copy link
Copy Markdown
Contributor Author

hostep commented Mar 22, 2026

Seems like it didn't fail on the first run, so I'd say that's a good sign this PR didn't cause a new problem

@engcom-Hotel engcom-Hotel added the Priority: P2 A defect with this priority could have functionality issues which are not to expectations. label Mar 24, 2026
@github-project-automation github-project-automation bot moved this to Pending Review in Pull Requests Dashboard Mar 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Priority: P2 A defect with this priority could have functionality issues which are not to expectations. Progress: pending review

Projects

Status: Pending Review

Development

Successfully merging this pull request may close these issues.

2 participants