Skip to content

docs(backend): add backend component overview section #12871

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
RATNAPRADEEP wants to merge 6 commits into
base: master
Choose a base branch
from
Open

docs(backend): add backend component overview section #12871

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ac30a2f
docs(backend): add backend component overview section
RATNAPRADEEP Feb 21, 2026
01bd9f3
Merge branch 'master' into docs-backend-overview
RATNAPRADEEP Feb 23, 2026
d3fb0a0
docs: add Scheduled Workflow Controller and Viewer Controller to back…
RATNAPRADEEP Feb 23, 2026
6df9265
fix: update driver description as per review
RATNAPRADEEP Mar 31, 2026
8f7d265
Merge branch 'master' into docs-backend-overview
RATNAPRADEEP Mar 31, 2026
624e43e
Merge branch 'master' into docs-backend-overview
RATNAPRADEEP Apr 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions backend/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,24 @@
This directory contains code for the components that comprise the Kubeflow
Pipelines backend.

## Backend Component Overview

The Kubeflow Pipelines backend is composed of several cooperating
components that manage pipeline execution and lifecycle:

- **API Server** – Exposes REST and gRPC endpoints for managing
pipelines, runs, and experiments.
- **Driver** – Translates pipeline specifications into executable
workflow definitions.
Copy link

Copilot AI Feb 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Driver description as "Translates pipeline specifications into executable workflow definitions" may be imprecise. Based on the code in backend/src/v2/driver/driver.go and backend/src/v2/cmd/driver/main.go, the driver appears to be responsible for orchestrating task execution (ROOT_DAG, DAG, CONTAINER types) and managing execution state with MLMD, rather than translating specifications.

The pipeline-to-workflow translation is typically done by the compiler component. Consider revising this description to something more accurate like "Orchestrates task execution and manages execution state" or "Prepares and manages the execution context for pipeline tasks".

Suggested change
- **Driver** – Translates pipeline specifications into executable
workflow definitions.
- **Driver** – Orchestrates task execution and manages execution state
with ML Metadata (MLMD).

Copilot uses AI. Check for mistakes.
- **Launcher** – Executes pipeline tasks within Kubernetes pods.
- **Persistence Agent** – Reconciles workflow state and updates
run status in the database.
- **Metadata Writer** – Handles interactions with ML Metadata (MLMD).
- **Cache Layer** – Manages execution result reuse when caching is enabled.

Understanding these components helps contributors identify where
new features, debugging logic, or lifecycle changes should be implemented.

This README will help you set up your coding environment in order to build and run the Kubeflow Pipelines backend. The KFP backend powers the core functionality of the KFP platform, handling API requests, workflow management, and data persistence.

## Prerequisites
Expand Down
Loading