Feature/build agent

[thomashebrard]

Summary by cubic

Adds a builder ops layer and a pipeline‑level dry run that returns a GraphSpec. Runner and CLI now support multi-bundle inputs via mthds_contents and bundle_uris, with unified graph generation and clearer errors.

• New Features

  • Builder ops: concept_ops, pipe_ops, inputs_ops, output_ops, models_ops, runner_code_ops, validate_ops.
  • dry_run_pipeline produces a GraphSpec from one or more contents; used by CLI graph and the API.
  • Agent run emits ReactFlow HTML and side‑effect metadata.

• Refactors

  • Renamed mthds_content→mthds_contents and bundle_uri→bundle_uris; auto-selects domain‑qualified main_pipe across multiple contents and skips duplicate loads.
  • Graph rendering delegates to dry_run_pipeline; agent inputs command delegates to builder ops.
  • TOML decode errors now include line/column; HTTP URL validation uses streaming GET; bumped mthds to >=0.2.0; updated agent error hint to reference mthds_contents.

^{Written for commit ce6b230. Summary will update on new commits.}

[cubic-dev-ai]

18 issues found across 22 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pipelex/builder/operations/output_ops.py">

<violation number="1" location="pipelex/builder/operations/output_ops.py:35">
P1: Re-opening a blank library here drops the libraries that `validate_bundle()` already loaded, so output generation can fail for bundles with external dependencies.</violation>

<violation number="2" location="pipelex/builder/operations/output_ops.py:43">
P1: `json.loads()` will crash for `output_format=PYTHON`, even though this helper accepts that enum value.</violation>
</file>

<file name="pipelex/graph/graph_rendering.py">

<violation number="1" location="pipelex/graph/graph_rendering.py:221">
P1: This new helper drops `bundle_uri`, so file-backed content passed with its `library_dirs` can load the same bundle twice and fail with a duplicate-domain error.</violation>
</file>

<file name="pipelex/builder/operations/concept_ops.py">

<violation number="1" location="pipelex/builder/operations/concept_ops.py:48">
P2: Guard non-dict structure values here so malformed specs become `ValidationError`s instead of crashing with `TypeError`.</violation>

<violation number="2" location="pipelex/builder/operations/concept_ops.py:81">
P1: List-typed structure fields lose `item_type`/`item_concept_ref` during TOML serialization.</violation>
</file>

<file name="pipelex/cli/agent_cli/commands/assemble_cmd.py">

<violation number="1" location="pipelex/cli/agent_cli/commands/assemble_cmd.py:58">
P1: This helper call removes the command’s `ConceptLoadError`/`PipeLoadError` wrapping, so TOML load failures now leak as raw exception types and lose the structured agent error fields.</violation>
</file>

<file name="pipelex/builder/operations/validate_ops.py">

<violation number="1" location="pipelex/builder/operations/validate_ops.py:61">
P2: Preserve the per-pipe dry-run status instead of hardcoding `SUCCESS`; otherwise allowed failures and skipped dry runs are reported as passing.</violation>

<violation number="2" location="pipelex/builder/operations/validate_ops.py:117">
P1: Use the returned `DryRunOutput.status` here; `dry_run_pipe(..., raise_on_failure=True)` can still yield `FAILURE` or `SKIPPED` without throwing.</violation>
</file>

<file name="pipelex/builder/operations/runner_code_ops.py">

<violation number="1" location="pipelex/builder/operations/runner_code_ops.py:39">
P1: Load the configured libraries before `load_from_blueprints`; otherwise runner generation breaks for bundles that depend on shared library definitions.</violation>

<violation number="2" location="pipelex/builder/operations/runner_code_ops.py:48">
P1: The generated runner does not load the bundle it was built from, so it can fail to resolve `pipe_code` at runtime.</violation>

<violation number="3" location="pipelex/builder/operations/runner_code_ops.py:48">
P1: Forward the pipe's output multiplicity here; otherwise generated runners call the scalar `main_stuff_as(...)` path for list outputs.</violation>
</file>

<file name="pipelex/builder/operations/models_ops.py">

<violation number="1" location="pipelex/builder/operations/models_ops.py:57">
P1: Backend filtering drops valid talent mappings because this lookup does not normalize `$...` / `preset:...` preset references before querying `presets_dict`.</violation>
</file>

<file name="pipelex/builder/operations/inputs_ops.py">

<violation number="1" location="pipelex/builder/operations/inputs_ops.py:46">
P1: Don't reopen a fresh library after validating inline bundle content; it drops external dependencies before `get_required_pipe()` runs.</violation>
</file>

<file name="pipelex/builder/operations/assemble_ops.py">

<violation number="1" location="pipelex/builder/operations/assemble_ops.py:51">
P1: Guard the path probe against `OSError`. Inline TOML that contains path-invalid text can currently fail on Windows before it ever reaches `tomlkit.parse()`.</violation>

<violation number="2" location="pipelex/builder/operations/assemble_ops.py:59">
P2: `main_pipe` is optional in the bundle blueprint, but this assembler makes it mandatory, so it cannot build valid bundles that omit that field.

(Based on your team's feedback about `main_pipe` being optional in `.mthds` bundles.) [FEEDBACK_USED]</violation>
</file>

<file name="pipelex/builder/operations/pipe_ops.py">

<violation number="1" location="pipelex/builder/operations/pipe_ops.py:29">
P2: Use the existing config-backed talent mappings instead of duplicating them here; otherwise CLI TOML generation can drift from the builder's actual preset resolution.

(Based on your team's feedback about reusing existing helpers instead of duplicating logic.) [FEEDBACK_USED]</violation>

<violation number="2" location="pipelex/builder/operations/pipe_ops.py:185">
P1: `PipeSearch` specs are parsed but never serialized, so generated TOML drops the required search fields.</violation>

<violation number="3" location="pipelex/builder/operations/pipe_ops.py:195">
P1: `PipeExtract` TOML generation drops `max_page_images`, `page_image_captions`, and `page_views`.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

P2: Guard non-dict structure values here so malformed specs become ValidationErrors instead of crashing with TypeError.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At pipelex/builder/operations/concept_ops.py, line 48:

<comment>Guard non-dict structure values here so malformed specs become `ValidationError`s instead of crashing with `TypeError`.</comment>

<file context>
@@ -0,0 +1,127 @@
+                }
+            else:
+                # Full field spec
+                field_data["the_field_name"] = field_name
+                converted_structure[field_name] = field_data
+        spec_data["structure"] = converted_structure
</file context>

[cubic-dev-ai]

1 issue found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pipelex/pipe_run/dry_run_pipeline.py">

<violation number="1" location="pipelex/pipe_run/dry_run_pipeline.py:39">
P2: This duplicates `execute_pipeline`'s bundle parsing instead of reusing the runner's existing `main_pipe` resolution, which adds a second parse and changes the missing-`main_pipe` error path.

(Based on your team's feedback about reusing existing helpers instead of duplicating logic.) [FEEDBACK_USED]</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pipelex/tools/misc/http_utils.py">

<violation number="1" location="pipelex/tools/misc/http_utils.py:44">
P3: This changes the validator from a documented HEAD probe to a GET request, so the function's contract is now misleading. Either restore HEAD here or update the public docstring/callers to reflect the new semantics.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

2 issues found across 19 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pyproject.toml">

<violation number="1" location="pyproject.toml:58">
P2: Pin the Git source to an immutable revision (`rev`) instead of a floating branch to keep builds reproducible.</violation>
</file>

<file name="pipelex/pipeline/pipeline_run_setup.py">

<violation number="1" location="pipelex/pipeline/pipeline_run_setup.py:175">
P1: Partial bundle overlap still reloads already-loaded blueprints, which can re-register domains and fail setup.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

1 issue found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pipelex/temporal/worker_cli.py">

<violation number="1" location="pipelex/temporal/worker_cli.py:52">
P2: `--project` is effectively ignored after this change because worker initialization no longer uses the resolved project name.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

P2: --project is effectively ignored after this change because worker initialization no longer uses the resolved project name.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At pipelex/temporal/worker_cli.py, line 52:

<comment>`--project` is effectively ignored after this change because worker initialization no longer uses the resolved project name.</comment>

<file context>
@@ -47,16 +48,8 @@ def configure(
-
-    # Force-enable Temporal when running as a worker, regardless of config
-    from pipelex.config import get_config  # noqa: PLC0415
+    Pipelex.make(temporal_enabled=True)
 
     if not get_config().temporal.is_enabled:
</file context>

[cubic-dev-ai]

3 issues found across 8 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pipelex/builder/operations/inputs_ops.py">

<violation number="1" location="pipelex/builder/operations/inputs_ops.py:46">
P2: `mthds_contents` now accepts multiple bundles, but only `blueprints[0]` is loaded into the new library. Pipes from later contents won’t be available, so `get_required_pipe` can fail even though `validate_bundle` succeeded. Load all blueprints when building the library.</violation>
</file>

<file name="pipelex/builder/operations/output_ops.py">

<violation number="1" location="pipelex/builder/operations/output_ops.py:32">
P2: `mthds_contents` is now a list, but the function still loads only `blueprints[0]`. Pipes defined in later bundle contents won’t be available, which can make `get_required_pipe` fail for valid inputs. Load all blueprints (or select the one containing `pipe_code`).</violation>
</file>

<file name="pipelex/builder/operations/validate_ops.py">

<violation number="1" location="pipelex/builder/operations/validate_ops.py:110">
P2: `validate_bundle_content` now accepts multiple `.mthds` contents but still returns only the first blueprint. This drops additional bundles silently when callers pass more than one content. Return all blueprints (or explicitly reject multi-content input) to match the new list input.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

3 issues found across 7 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pipelex/builder/operations/validate_ops.py">

<violation number="1" location="pipelex/builder/operations/validate_ops.py:81">
P2: Dry-run statuses are looked up with `the_pipe.code`, but dry-run results are keyed by `pipe_ref`; this can silently misreport failed/skipped pipes as SUCCESS.</violation>

<violation number="2" location="pipelex/builder/operations/validate_ops.py:112">
P2: This lookup also uses `the_pipe.code` instead of `pipe_ref`, so `validated_pipes` can incorrectly show SUCCESS for non-success dry runs.</violation>
</file>

<file name="pipelex/builder/operations/assemble_ops.py">

<violation number="1" location="pipelex/builder/operations/assemble_ops.py:55">
P2: Catching `OSError` around file opening hides real file-read failures and incorrectly falls back to parsing the path string as inline TOML.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

1 issue found across 13 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pipelex/builder/operations/models_ops.py">

<violation number="1" location="pipelex/builder/operations/models_ops.py:116">
P2: Backend filtering still drops alias-based talent mappings because it only looks up stripped names in `presets_dict`.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}