Heroku infinite deployment attempt even on failure

[yathomasi]

We have been getting infinite deployment issue with heroku on all of our websites

• Exp sharing updates treeverse/dvc.org#4485
• exp sharing updates - Studio docs treeverse/dvc.org#4487
• https://github.com/iterative/iterative.ai/pull/839

I opened a ticket on Heroku Support in addition to this to research more and the Heroku team replied it could be because we have Wait for CI to pass. So upon digging further, it turns out we have a loop because of Wait for CI to pass in Heroku and link check GHA workflow which gets triggered on deployment status. Since it's getting triggered even on failure causing the change in CI which in turn triggers the Heroku.

There is a few more test I am doing

• Test: Heroku deployment issue treeverse/dvc.org#4525
• Test: Heroku deployment issue treeverse/dvc.org#4526
• Will also try to address https://github.com/iterative/iterative.ai/issues/810

[shcheklein]

it could be because we have Wait for CI to pass

When enabled, new review apps will wait to deploy until Heroku CI passes.. Do we actually need this? Is it equivalent of waiting for linters and GH CI to pass?

[yathomasi]

Is it equivalent of waiting for linters and GH CI to pass?

Yes, we have been using it that way so that we can fail a few PRs(like renovate or branch with certain phrases) and it would halt the deployment. It was a hacky implementation as there is no simple way Heroku provides us to limit a few PRs from deployment.

I tried a few different ways to see if skipping GHA that gets triggered based on deployment_status and allowed it only when it's necessary. But, it still triggered the issue.

I will try to see with Heroku Support to see any way we can fix this. Otherwise, I think we should proceed by disabling the Wait for CI to pass and instead further limit the no of renovate PRs and maybe schedule monthly.