Feature: Support Advanced Plan Metadata (Sub-agents, Tool Calls, MCP)

[jellydn]

Feature Request: Support Advanced Plan Metadata (Sub-agents, Tool Calls, MCP)

What

Extend Plannotator to capture and display rich metadata about the planning process, including:

• Sub-agent invocations: Which sub-agents were spawned (Explore, Plan, code-reviewer, etc.)
• Tool calls: Tools considered or called during planning (Read, Grep, Bash, etc.)
• MCP server interactions: Which MCP servers were queried and what data was retrieved
• Decision rationale: Why certain approaches were chosen over alternatives
• Planning iterations: How many times the plan was refined

Why

Currently, Plannotator only captures the final markdown plan content (event.tool_input?.plan). This is like seeing the final destination without knowing the journey. Users approving plans miss critical context:

1. Informed Decision Making: Understanding HOW Claude arrived at a plan helps users evaluate its quality. A plan that spawned 5 sub-agents and queried 3 MCP servers demonstrates thoroughness.

2. Trust Building: Showing the research process (files read, code explored) builds user confidence that the plan is well-informed.

3. Debugging Planning Issues: When a plan misses something, seeing which tools/sub-agents were NOT used helps identify gaps.

4. Learning Opportunity: Users can learn Claude's planning patterns by seeing which sub-agents handle which types of problems.

5. Approval Granularity: Users might want to approve the "research" phase but request revisions on the "implementation" phase.

How

Phase 1: Data Capture (Core)

1.1 Extend Hook Event Parsing

Update apps/hook/server/index.ts to extract additional fields from the hook event:

// Current: Only captures plan markdown
planContent = event.tool_input?.plan || "";

// Proposed: Capture rich metadata
interface PlanMetadata {
  plan: string;                    // Existing markdown
  subAgents?: SubAgentInvocation[]; // NEW
  toolCalls?: ToolCallSummary[];    // NEW
  mcpInteractions?: MCPInteraction[]; // NEW
  iterations?: number;              // NEW
  planningDuration?: number;        // NEW (milliseconds)
  rationale?: string;               // NEW (why this approach)
}

interface SubAgentInvocation {
  name: string;              // "Explore", "Plan", "code-reviewer"
  purpose: string;           // Why this agent was spawned
  status: "completed" | "failed" | "timeout";
  tokensUsed?: number;
  keyFindings?: string[];    // One-sentence summaries
}

interface ToolCallSummary {
  tool: string;              // "Read", "Grep", "Bash"
  purpose: string;           // What the tool was looking for
  files?: string[];          // Files read/searched
  commands?: string[];       // Bash commands run
  resultCount?: number;      // Number of results
}

interface MCPInteraction {
  server: string;            // MCP server name
  tool: string;              // Tool called on server
  purpose: string;           // Why this MCP was needed
  success: boolean;
}

1.2 Update Type Definitions

Add to packages/ui/types.ts:

export interface PlanMetadata {
  plan: string;
  subAgents?: SubAgentInvocation[];
  toolCalls?: ToolCallSummary[];
  mcpInteractions?: MCPInteraction[];
  iterations?: number;
  planningDuration?: number;
  rationale?: string;
}

export interface SubAgentInvocation {
  name: string;
  purpose: string;
  status: "completed" | "failed" | "timeout";
  tokensUsed?: number;
  keyFindings?: string[];
}

export interface ToolCallSummary {
  tool: string;
  purpose: string;
  files?: string[];
  commands?: string[];
  resultCount?: number;
}

export interface MCPInteraction {
  server: string;
  tool: string;
  purpose: string;
  success: boolean;
}

1.3 Server API Updates

Modify apps/hook/server/index.ts:

// API: Get plan with metadata
if (url.pathname === "/api/plan") {
  return Response.json({
    plan: planContent,
    metadata: planMetadata,  // NEW: Rich metadata
  });
}

Phase 2: UI Components (Display)

2.1 Planning Insights Panel

Create new component packages/ui/components/PlanningInsights.tsx:

interface PlanningInsightsProps {
  metadata: PlanMetadata;
}

export const PlanningInsights: React.FC<PlanningInsightsProps> = ({ metadata }) => {
  return (
    <div className="planning-insights-panel">
      {/* Sub-agents section */}
      {metadata.subAgents && (
        <Section title="Sub-agents Spawned" icon={branchIcon}>
          {metadata.subAgents.map(agent => (
            <AgentCard key={agent.name} agent={agent} />
          ))}
        </Section>
      )}

      {/* Tool calls section */}
      {metadata.toolCalls && (
        <Section title="Tools Used" icon={toolIcon}>
          <ToolCallList tools={metadata.toolCalls} />
        </Section>
      )}

      {/* MCP interactions section */}
      {metadata.mcpInteractions && (
        <Section title="MCP Servers Queried" icon={serverIcon}>
          <MCPList interactions={metadata.mcpInteractions} />
        </Section>
      )}

      {/* Planning stats */}
      <PlanningStats
        iterations={metadata.iterations}
        duration={metadata.planningDuration}
      />
    </div>
  );
};

2.2 Integrate into Main App

Update packages/editor/App.tsx:

const [metadata, setMetadata] = useState<PlanMetadata | null>(null);

// Fetch from API
useEffect(() => {
  fetch('/api/plan')
    .then(res => res.json())
    .then((data: { plan: string; metadata?: PlanMetadata }) => {
      setMarkdown(data.plan);
      setMetadata(data.metadata || null);
    });
}, []);

// Add collapsible insights panel
{metadata && (
  <PlanningInsights
    metadata={metadata}
    isOpen={showInsights}
    onToggle={() => setShowInsights(!showInsights)}
  />
)}

Phase 3: Advanced Features

3.1 Interactive Metadata Filtering

Allow users to filter plan content by metadata:

• "Show only steps involving the Explore agent"
• "Highlight sections that used Grep tool"
• "Display MCP-derived insights separately"

3.2 Export with Metadata

Extend exportDiff to include planning context:

export function exportWithMetadata(
  blocks: Block[],
  annotations: Annotation[],
  metadata: PlanMetadata
): string {
  let output = exportDiff(blocks, annotations);

  // Append planning context
  output += "\n\n## Planning Context\n\n";
  output += `- **Sub-agents**: ${metadata.subAgents?.map(a => a.name).join(", ") || "none"}\n`;
  output += `- **Tools used**: ${metadata.toolCalls?.map(t => t.tool).join(", ") || "none"}\n`;
  output += `- **MCP servers**: ${metadata.mcpInteractions?.map(m => m.server).join(", ") || "none"}\n`;
  output += `- **Planning iterations**: ${metadata.iterations || 1}\n`;

  return output;
}

3.3 URL Sharing with Metadata

Update packages/ui/utils/sharing.ts to compress metadata in share URLs (may require aggressive compression due to size limits).

3.4 Obsidian Integration

Save planning metadata to Obsidian as YAML frontmatter:

---
sub_agents: [Explore, Plan]
tools_used: [Read, Grep, Bash]
mcp_servers: [context7, memory-server]
planning_iterations: 3
---

File Change Summary

Modified Files:

• apps/hook/server/index.ts - Parse and serve metadata
• packages/ui/types.ts - Add metadata interfaces
• packages/editor/App.tsx - Display metadata panel
• packages/ui/utils/sharing.ts - Include metadata in shares
• packages/ui/utils/parser.ts - Export with metadata context

New Files:

• packages/ui/components/PlanningInsights.tsx - Main metadata display component
• packages/ui/components/AgentCard.tsx - Sub-agent invocation card
• packages/ui/components/ToolCallList.tsx - Tool calls visualization
• packages/ui/components/MCPList.tsx - MCP interaction list

Open Questions

1. Data Availability: Does the ExitPlanMode hook event actually contain this metadata, or does it only provide the final plan markdown? Need to inspect actual hook event structure.

2. Size Limits: If metadata is large, can it still fit in URL sharing? May need server-side storage with short IDs.

3. Privacy: Some tool calls may include sensitive data (file paths, command output). Need to filter or sanitize.

Acceptance Criteria

• Hook server captures and serves planning metadata (sub-agents, tools, MCP)
• UI displays collapsible "Planning Insights" panel
• Each sub-agent invocation shows: name, purpose, status, key findings
• Tool calls are grouped by type with file/command counts
• MCP interactions show server name, tool, and success status
• Metadata is included in export diff output
• Metadata appears in Obsidian frontmatter when saved
• Panel is toggleable and collapsed by default (don't distract from plan content)
• URL sharing includes metadata OR shows warning that metadata is excluded
• Performance impact is minimal (metadata rendering < 100ms)

Dependencies

• Claude Code Hook Event Structure: Need documentation on what data is actually available in ExitPlanMode events
• Potential Hook API Changes: May require Anthropic to expose additional metadata in hook events

Related Issues

• Feature: Add Image Upload Support for Annotations #18 (Image Annotations) - Both issues enhance plan review with richer context

[backnotprop]

Implementation Notes: Phased Approach

After investigating the data availability in both Claude Code and OpenCode, here's our planned approach.

What's Actually Available

The hook events themselves are thin - they only contain the final plan markdown. However, the session storage contains rich history that we can query:

Platform	Storage Location	Session ID in Hook?
Claude Code	~/.claude/projects/{encoded-path}/{session}.jsonl	❌ No - need to infer
OpenCode	~/.local/share/opencode/storage/	✅ Yes

Phased Implementation

Phase 1: Files Read + Initiating Message (first target)

• Display all files accessed via Read tool during planning
• Show the user message that triggered plan mode
• Show all user messages within plan mode

Phase 2: Tool Call Summary

• Count of each tool type used (Read, Grep, Glob, Bash, etc.)
• Searchable file patterns (from Grep/Glob)

Phase 3: Sub-agent Context (if data is available)

• Which sub-agents were spawned (Explore, etc.)
• Their prompts and key findings

Technical Approach

Claude Code:

• Parse session JSONL file
• Find all events between EnterPlanMode and ExitPlanMode tool calls
• Extract Read tool inputs for file list
• Walk parentUuid chain backward from EnterPlanMode to find initiating user message

OpenCode:

• Query message storage using sessionID from event
• Filter messages/parts where agent === "plan"
• Extract tool parts with tool === "read" for file list
• Find first user message where agent transitions to "plan"

Open Questions

1. Claude Code session discovery: Hook doesn't provide sessionId. Options being explored:

   • Infer from CWD + encoded project path
   • Read most recently modified session file
   • Check for environment variables

2. UI placement: Considering a collapsible "Planning Context" panel, collapsed by default to not distract from plan content.

3. Performance: Long sessions may have large JSONL files. May need to only parse from EnterPlanMode forward rather than full file.

---

This addresses a subset of your original feature request - focusing on what's reliably available rather than the full metadata suite. The sub-agent/MCP/rationale data would require changes to what Claude Code/OpenCode expose in their hook events.

[backnotprop]

Update: Hook Event Data Confirmed

We ran a debug hook to capture all events during a planning session. Great news: the data we need is available.

What's in the PermissionRequest Event for ExitPlanMode

{
  "session_id": "e610b2ac-38d6-48cb-b292-d61d9764a4b5",
  "transcript_path": "/Users/ramos/.claude/projects/.../session.jsonl",
  "cwd": "/Users/ramos/cupcake/cupcake-rego/planning-hook",
  "permission_mode": "plan",
  "hook_event_name": "PermissionRequest",
  "tool_name": "ExitPlanMode",
  "tool_input": { "plan": "..." }
}

Key finding: transcript_path is provided directly in the hook event. No need to infer or discover the session file.

Other Useful Events Captured

Event	Key Data
UserPromptSubmit	Full user prompt text
PreToolUse	tool_name, tool_input (e.g., file paths for Read)
PostToolUse	Same + tool_response with full results
SubagentStop	agent_id, agent_transcript_path

Revised Implementation Plan

Phase 1: Planning Context Panel

Since we have transcript_path, Plannotator can:

1. Parse the session JSONL file
2. Find all events between EnterPlanMode and ExitPlanMode
3. Extract:
   • Files read (Read tool calls → tool_input.file_path)
   • User messages during planning
   • The initiating user message (walk backward from EnterPlanMode)

Phase 2: Tool Call Summary

• Count by tool type (Read, Grep, Glob, Bash, etc.)
• Grep/Glob patterns searched

Phase 3: Sub-agent Context

• Parse agent_transcript_path files from SubagentStop events
• Show which sub-agents ran and their findings

No Blockers

The original concern about session discovery is resolved. Implementation can proceed.

[jellydn]

Great, thanks for looking into this request. Here is an example of what I am thinking how it looks like: