fix: Add deploy concurrency option and preserve default work pool behavior for multi-deploy#22326
fix: Add deploy concurrency option and preserve default work pool behavior for multi-deploy#22326hemantmm wants to merge 5 commits into
Conversation
…avior for multi-deploy closes: PrefectHQ#19096
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e9cb92a09d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| await asyncio.gather( | ||
| *(deploy_with_limit(config) for config in validated_configs), | ||
| ) |
There was a problem hiding this comment.
Preserve fail-fast behavior when concurrency is one
With the default concurrency=1, this schedules every deployment coroutine up front; if one _run_single_deploy raises, asyncio.gather propagates the exception but does not cancel the other awaitables, and releasing the semaphore can let the next config enter _run_single_deploy before the CLI exits. In a multi-deploy where an earlier deployment fails, later deployments can still be created/updated, regressing the previous sequential fail-fast behavior and causing unexpected partial deploys after the reported failure.
Useful? React with 👍 / 👎.
Merging this PR will not alter performance
Comparing Footnotes
|
Signed-off-by: Hemant M Mehta <hemant29mehta@gmail.com>
Signed-off-by: Hemant M Mehta <hemant29mehta@gmail.com>
Signed-off-by: Hemant M Mehta <hemant29mehta@gmail.com>
Signed-off-by: Hemant M Mehta <hemant29mehta@gmail.com>
closes: #19096
Summary
This PR introduces a
--deploy-concurrencyoption toprefect deployto control the number of deployments created concurrently when deploying multiple deployments.Additionally, it preserves the existing behavior of applying the configured default work pool to deployment configurations that do not explicitly specify one, while avoiding unnecessary warnings when the default pool originates from settings.
Checklist
<link to issue>"mint.json.