Skip to content

Add in repeatable migration batching process#226

Open
nickpileggi947 wants to merge 1 commit intoSnowflake-Labs:masterfrom
nickpileggi947:feature/batch_deploy
Open

Add in repeatable migration batching process#226
nickpileggi947 wants to merge 1 commit intoSnowflake-Labs:masterfrom
nickpileggi947:feature/batch_deploy

Conversation

@nickpileggi947
Copy link
Copy Markdown

Add in repeatable migration batching process. This allows for repeatable migrations to be batched together to speed up deployment. The number of migrations batched together can be controlled via the config file and params. In the event of a failure in one of the batched repeatable migrations the repeatable migrations will be retried until the affected migration is found.

…e repeatable migrations to be batched together to speed up deployment. The number of migrations batched together can be controlled via the config file and params
@sfc-gh-tmathew sfc-gh-tmathew added enhancement New feature or request Under Review This is being discussed without planned changes labels Mar 6, 2024
@nickpileggi947
Copy link
Copy Markdown
Author

Any feedback on this/any desire to merge this in if I resolve conflicts from the latest changes

@sfc-gh-tmathew sfc-gh-tmathew added the community-contribution Submitted by community label Nov 18, 2025
@sfc-gh-tmathew
Copy link
Copy Markdown
Collaborator

sfc-gh-tmathew commented Feb 9, 2026

Hi @nickpileggi947,

Thank you for this PR to add repeatable migration batching for faster deployments.

This PR has been open for about 2 years and likely has significant conflicts. The batching concept is interesting, but we'd need to understand:

  1. What performance improvement did you observe with batching?
  2. How does the retry-on-failure logic work when a batch fails?

Questions:

  1. Are you still interested in pursuing this feature?
  2. If so, would you be able to rebase against master?

If we don't hear back in the next few weeks, we may close this PR to keep our backlog manageable. You're welcome to reopen later.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution Submitted by community enhancement New feature or request Under Review This is being discussed without planned changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants