Replace single flow.yaml with distributed YAML configuration system#24035
Draft
lucia-sb wants to merge 22 commits into
Draft
Replace single flow.yaml with distributed YAML configuration system#24035lucia-sb wants to merge 22 commits into
lucia-sb wants to merge 22 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b294401a27
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Comment on lines
+49
to
+53
| discover_and_register_phases( | ||
| self._phase_registry, | ||
| Path(__file__).parent.parent / "phases", | ||
| "ddev.ai.phases", | ||
| ) |
There was a problem hiding this comment.
When a distributed flow references a custom phase class, this initialization now only registers classes from ddev.ai.phases; it no longer scans the flow's own phases/ directory as the old orchestrator did. The existing OpenMetrics custom phase lives at ddev/src/ddev/ai/flows/openmetrics/phases/inspect_endpoint.py, so a flow entry with class: InspectEndpointPhase reaches self._phase_registry.get(...) and fails as an unknown phase before the pipeline can start.
Useful? React with 👍 / 👎.
…wConfig, envelopes) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…validation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…igurationEngine Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…g from phases/config; update all import sites Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…trator pending Task 5 - phases/config.py: rename _detect_cycles -> detect_cycles (public API) - config/engine.py: update call site to detect_cycles - tests/ai/config/test_engine.py: add missing engine tests - runtime/orchestrator.py: stub on_initialize with FlowConfigError pending Task 5 migration; remove unused imports Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…d absolute paths Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lucia-sb
force-pushed
the
lucia-sb/ai-6835/dist-config
branch
from
b294401 to
0bb6525
Compare
This comment has been minimized.
This comment has been minimized.
…tests, expand test coverage - Extract FlowConfigError and detect_cycles into config/errors.py; phases/config.py becomes a compat shim - Move TaskConfig, CheckpointConfig, FlowEntry into config/models.py; replace mutable defaults with Field(default_factory=...) - Rewrite engine.py: _VarState dataclass, _deduplicate(), seen_files dedup, match-based _registry_for(), phase_discovery_targets(), only insert registry entry when no conflict; fix pending key type to ResourceKind - Rewrite orchestrator.py: split on_initialize() into 4 helpers; _build_runtime() returns 4-tuple; wrap _agent_logger.close() in try/except in on_finalize() - agentic_phase.py: replace cast() with assert; add assert for system_prompt_path; import from config.errors/config.models - goal.py: add _ReviewerOutput model + TypeAdapter; replace manual JSON parse with _REVIEWER_ADAPTER.validate_json() - inspect_endpoint.py: remove unnecessary try/except around unlink(missing_ok=True); use Path.replace() not os.replace() - test_engine.py: fix Windows expanduser (set USERPROFILE); fix match string for nonexistent path; add multi-dir merge, user-dir override, overlapping dir dedup, detect_cycles direct tests, _parse_file failure modes, variable resolution, path resolution, conflict source ordering tests - test_models.py: add test_resource_envelope_unknown_type_raises Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…x strict bool, fix test condition - inspect_endpoint.py: restore import os and os.replace() (test patches inspect_endpoint.os); wrap unlink() in try/except OSError to handle directory case silently - goal.py: add strict=True to _ReviewerOutput ConfigDict to prevent Pydantic coercing non-bool JSON values to bool - test_engine.py: fix test_build_flow_error_paths condition from 'conflict' in match to match == 'conflicts' to avoid triggering the extra-file setup for variable_default_conflict Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
Validation ReportAll 21 validations passed. Show details
|
luisorofino
force-pushed
the
loa/task-context
branch
from
7c0acc6 to
36f6665
Compare
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Replaces the single
flow.yamlfile model with a distributed YAML configuration system. Typed resource objects (agents, phases, flows) can now be spread across multiple YAML files in any directory and are assembled at runtime by a newConfigurationEngine.Motivation
The monolithic
flow.yamlapproach doesn't scale — as the number of flows, agents, and phases grows, a single file becomes hard to maintain and prevents reuse across flows. A Terraform-style distributed model lets each resource live in its own file, be shared across flows, and be overridden by user-supplied directories.Key changes:
ddev/src/ddev/ai/config/package withmodels.py(resource types + envelopes) andengine.py(ConfigurationEnginewith multi-directory scanning andbuild_flowvalidation)AgentConfigandPhaseConfigmoved fromddev.ai.phases.configtoddev.ai.config.models;FlowConfigreplaced by distributed envelope modelconfig_dirremoved fromFlowContext; paths are pre-resolved to absolute bybuild_flow()before reaching phasesPhaseOrchestratorupdated to acceptConfigurationEngine + flow_nameinstead of a single file pathsrc/ddev/ai/flowsadded to wheel artifacts so bundled flow definitions ship with the packageReview checklist (to be filled by reviewers)
qa/requiredif this PR needs QA validation, orqa/skip-qaif it does not. Exactly one of the two is required.backport/<branch-name>label to the PR and it will automatically open a backport PR once this one is merged