Skip to content

tests-ui.yml github action workflow fails unexpectedly intermittently #4040

Description

@jgravois

lately i've noticed more intermittent flakiness in tests.ui.yml that i've been able to workaround via manually rerunning the workflow.

i took a quick look at the lighthouse docs and noticed that our attempt to trigger 3 runs isn't being honored, likely because the configuration syntax changed after @machikoyasuda last touched this back in #202.

"ci": {
"numberOfRuns": 3,

Our options

  1. start triggering 3 runs again (as proposed in fix: set lighthouse numberOfRuns correctly to ensure gh actions honors it #4039).

to be perfectly honest its possible that one failure out of three will still fail the entire job, but it'd be a cheap test to just merge it and 'wait and see'. the only drawback being that the total runtime increases ~1 minute.

  1. continue to run lighthouse only once, remove the unused configuration setting, and tolerate/ignore the intermittent errors

  2. put some effort into diagnosing why calls to / and /help in the container are failing more often than they used to. is it a timing thing? an external dependency that doesn't load consistently?

Acceptance Criteria

  • tests-ui.yml passes in gh action runs associated with unrelated PRs 100% of the time.
  • our lighthouse config file contains no ignored/invalid declarations

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    choreInternal maintenance or code quality worktaskCross-functional or operational work such as documentationtestsRelated to automated testing (unit, UI, integration, etc.)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions