|
| 1 | +<!-- |
| 2 | +Copyright 2026 Josh Waldrep |
| 3 | +SPDX-License-Identifier: Apache-2.0 |
| 4 | +--> |
| 5 | + |
| 6 | +# Detection integration |
| 7 | + |
| 8 | +Pipelock is a real-time agent firewall. It blocks what it can see on the wire. |
| 9 | +That is a hard, narrow job, and it is not the whole detection story. |
| 10 | + |
| 11 | +Long-running agents move through multi-week attack chains, reason in |
| 12 | +natural language, and produce actions that look benign in isolation. |
| 13 | +No real-time gateway will catch all of that. The gate is the first |
| 14 | +line, not the last. |
| 15 | + |
| 16 | +This guide is for people building the layer that runs behind the gate. |
| 17 | +SIEM engineers, SOC analysts, and researchers training detection models |
| 18 | +all need the same thing upstream: structured, tamper-evident evidence of |
| 19 | +what the agent actually did. When receipt signing is enabled, Pipelock |
| 20 | +emits that evidence as signed action receipts. This guide covers how to |
| 21 | +consume them. |
| 22 | + |
| 23 | +## Real-time gateways are not enough |
| 24 | + |
| 25 | +Picture a malicious MCP server that runs a two-step attack across a |
| 26 | +week. |
| 27 | + |
| 28 | +Step one, Monday: the agent calls an innocent-looking tool. The tool |
| 29 | +response tells the agent to install a new hook that exfiltrates data |
| 30 | +to `https://google.com/report`. To an inline detector, that looks |
| 31 | +like a bad tool response, but the destination is benign. The |
| 32 | +content might sail through, especially under a looser policy. |
| 33 | + |
| 34 | +Step two, the following Monday: a different tool response tells the |
| 35 | +agent to change the hook's destination. To an inline detector, that |
| 36 | +request looks like a one-line config edit. The agent already has |
| 37 | +the hook, so changing an endpoint is not structurally suspicious. |
| 38 | +A real-time gateway viewing only that single moment has no reason |
| 39 | +to block it. |
| 40 | + |
| 41 | +Put the two steps next to each other and the intent is obvious. A |
| 42 | +detector looking at a week of activity can see the shape. A detector |
| 43 | +looking at a single request cannot. |
| 44 | + |
| 45 | +Long-window detection is the only way to catch attacks that are |
| 46 | +designed to look like two benign things. The question is what you |
| 47 | +feed that detector. |
| 48 | + |
| 49 | +## The primitive: signed action receipts |
| 50 | + |
| 51 | +When `flight_recorder.signing_key_path` is set in the Pipelock |
| 52 | +config, every proxy decision produces a signed action receipt. |
| 53 | +Receipts are Ed25519-signed, JSON-structured, and linked into a |
| 54 | +SHA-256 hash chain so any deletion or reordering is detectable |
| 55 | +after the fact. Without a signing key configured, Pipelock still |
| 56 | +enforces, and the flight recorder can still write other evidence |
| 57 | +entries, but the signed receipt stream is not produced. |
| 58 | + |
| 59 | +Generate a key with `pipelock keygen <name>`, set |
| 60 | +`flight_recorder.signing_key_path`, and start or restart Pipelock. |
| 61 | +If you replace the key file contents at the same configured path, |
| 62 | +reload will re-read that file. Changing the configured path still |
| 63 | +requires a restart. |
| 64 | + |
| 65 | +A receipt carries the fields a downstream detector needs to reason |
| 66 | +about the decision: |
| 67 | + |
| 68 | +| Field | What | |
| 69 | +|-------|------| |
| 70 | +| `action_id` | UUIDv7, unique per decision. Stable identifier for the record. | |
| 71 | +| `timestamp` | RFC 3339 wall-clock time the decision was made. | |
| 72 | +| `verdict` | `block`, `warn`, `exemption`, `allow`, `strip`, `redirect`, or `ask`. Deterministic. | |
| 73 | +| `layer` | Which scanner triggered (`mcp_response_scan`, `dlp_header`, `response_scan`, `airlock`, etc.). | |
| 74 | +| `pattern` | Named rule inside the layer (e.g., `Prompt Injection`, `aws_access_key`). | |
| 75 | +| `transport` | `fetch`, `forward`, `websocket`, `mcp_stdio`, `mcp_http_upstream`, `mcp_http_listener`, `connect`, `intercept`. | |
| 76 | +| `session_id` | Groups receipts from the same agent session. | |
| 77 | +| `principal` / `actor` | Who initiated the action and who enforced it. | |
| 78 | +| `policy_hash` | SHA-256 of the canonical policy config at decision time. Changes whenever the policy changes. | |
| 79 | +| `side_effect_class` / `reversibility` | Classification of the attempted action. | |
| 80 | +| `chain_prev_hash` / `chain_seq` | Hash-chain linkage to the prior receipt in the stream. | |
| 81 | +| `signature` / `signer_key` | Ed25519 signature and public key. | |
| 82 | + |
| 83 | +These are fixed fields with a fixed schema. A detector parses them |
| 84 | +with a JSON reader, not a log regex. |
| 85 | + |
| 86 | +Full canonical schema and field reference: |
| 87 | +<https://pipelab.org/learn/action-receipt-spec/> |
| 88 | + |
| 89 | +## Three downstream consumers |
| 90 | + |
| 91 | +The same receipt stream serves three different detection styles |
| 92 | +without modification. |
| 93 | + |
| 94 | +### SIEM rules |
| 95 | + |
| 96 | +Ship the flight-recorder JSONL file to Splunk, Datadog, Elastic, or |
| 97 | +any SIEM that ingests JSON. The standard pattern is a file shipper |
| 98 | +(Filebeat, Fluent Bit, Vector, or equivalent) tailing |
| 99 | +`flight_recorder.dir` and forwarding each line. Filter entries to |
| 100 | +`type == "action_receipt"` at the shipper or at the SIEM. |
| 101 | + |
| 102 | +The receipt fields map cleanly to structured search: group by |
| 103 | +`detail.action_record.action_id` or by `session_id` to reconstruct |
| 104 | +an agent's behavior, filter on `detail.action_record.verdict=block` |
| 105 | +to audit enforcement events, alert on |
| 106 | +`detail.action_record.pattern=aws_access_key` for secret-exfil |
| 107 | +attempts. |
| 108 | + |
| 109 | +Pipelock also ships a separate real-time emit pipeline (webhook, |
| 110 | +syslog, OTLP) that streams a different envelope format focused on |
| 111 | +security events and severity levels. That pipeline is for operator |
| 112 | +alerting, not for forwarding the full receipt chain. See |
| 113 | +[`siem-integration.md`](siem-integration.md) for that format. |
| 114 | +Receipts and emit events are complementary streams, not the same |
| 115 | +stream in different wrappers. |
| 116 | + |
| 117 | +### Analyst review |
| 118 | + |
| 119 | +When a SIEM rule fires or an agent's session looks suspicious, an |
| 120 | +analyst can pull the full receipt stream for that `session_id` and |
| 121 | +reconstruct every decision in order. The hash chain confirms the |
| 122 | +stream has not been edited since it was written. The `policy_hash` |
| 123 | +confirms which policy version was in force. The `signature` |
| 124 | +confirms the record came from Pipelock and not from a tampered |
| 125 | +agent log. |
| 126 | + |
| 127 | +This is the audit-trail use case. Receipts are designed to be |
| 128 | +presentable to a third party (auditor, incident responder, internal |
| 129 | +review) without requiring trust in Pipelock itself. The signature |
| 130 | +and the chain are the trust anchors. |
| 131 | + |
| 132 | +### Long-window LLM detection |
| 133 | + |
| 134 | +If your detection strategy involves feeding agent history to a |
| 135 | +reasoning model and asking "is this suspicious", what you feed the |
| 136 | +model matters more than which model you pick. Raw log text lets an |
| 137 | +attacker shape the model's view by shaping the agent's reasoning |
| 138 | +tokens. Structured receipts do not. |
| 139 | + |
| 140 | +A receipt stream is a deterministic sequence of fixed-field events. |
| 141 | +`verdict`, `layer`, `pattern`, `policy_hash`, and `timestamp` are |
| 142 | +not up for attacker manipulation. The attacker cannot inject text |
| 143 | +into the stream that changes what these fields say. The stream is |
| 144 | +signed and chain-linked, so injection is detectable. |
| 145 | + |
| 146 | +This is the architecture Zack Korman gestures at in his April 2026 |
| 147 | +video on AI agent threat detection: a funnel of cheaper LLMs |
| 148 | +filtering events, stronger LLMs confirming, and an agentic layer |
| 149 | +trying to disprove the finding. Whether that funnel works is its |
| 150 | +own question. But if it does, it only works on inputs the attacker |
| 151 | +cannot massage. Structured receipts are that kind of input. Raw |
| 152 | +reasoning tokens are not. |
| 153 | + |
| 154 | +## Worked example |
| 155 | + |
| 156 | +The [`tool-response-injection`](/examples/tool-response-injection/) |
| 157 | +example ships with Pipelock and runs the whole loop end-to-end. It |
| 158 | +uses a deliberately malicious MCP server that returns a prompt- |
| 159 | +injection payload disguised as a game result. |
| 160 | + |
| 161 | +Run the harness: |
| 162 | + |
| 163 | +```bash |
| 164 | +cd examples/tool-response-injection |
| 165 | +python3 demo.py |
| 166 | +``` |
| 167 | + |
| 168 | +The harness produces three artifacts worth looking at: |
| 169 | + |
| 170 | +1. **`evidence/evidence-proxy-0.jsonl`**: the MCP stdio evidence file. |
| 171 | + It contains the signed receipt stream as `action_receipt` entries |
| 172 | + and may also contain other recorder entries such as checkpoints. |
| 173 | +2. **`evidence-proxy-0.jsonl` from the HTTP upstream run**: same |
| 174 | + event shape, different `transport` field. |
| 175 | +3. **`signing.key.pub` hex output**: the public key printed to |
| 176 | + stdout, the only thing a third party needs to verify the stream. |
| 177 | + |
| 178 | +The harness verifies the stream inline with Python. For an |
| 179 | +independent check, install the reference verifier: |
| 180 | + |
| 181 | +```bash |
| 182 | +pip install pipelock-verify |
| 183 | +python -m pipelock_verify evidence/evidence-proxy-0.jsonl --key <public-key-hex> |
| 184 | +``` |
| 185 | + |
| 186 | +Or use the Go CLI that ships with Pipelock: |
| 187 | + |
| 188 | +```bash |
| 189 | +pipelock verify-receipt evidence/evidence-proxy-0.jsonl --key <public-key-hex> |
| 190 | +``` |
| 191 | + |
| 192 | +Both exit 0 on success, 1 on any signature failure, chain break, or |
| 193 | +reordering. The verifiers are byte-for-byte equivalent. |
| 194 | + |
| 195 | +### Consuming receipts in a detector |
| 196 | + |
| 197 | +Once the stream verifies, a downstream detector reads it like any |
| 198 | +JSONL source. This is a complete working example: |
| 199 | + |
| 200 | +```python |
| 201 | +"""Verify a pipelock receipt stream, then route each verified receipt |
| 202 | +to a pluggable handler for whatever downstream detector runs on. |
| 203 | +
|
| 204 | +Requires: pip install pipelock-verify |
| 205 | +Run: python verify_and_route.py evidence.jsonl <public-key-hex> |
| 206 | +""" |
| 207 | + |
| 208 | +import json |
| 209 | +import subprocess |
| 210 | +import sys |
| 211 | +from typing import Callable, Iterator |
| 212 | + |
| 213 | + |
| 214 | +def verified_receipts(path: str, pubkey_hex: str) -> Iterator[dict]: |
| 215 | + """Yield each receipt only if the full stream verifies. |
| 216 | +
|
| 217 | + Evidence files can contain non-receipt entries (checkpoints, other |
| 218 | + event types). We filter for type == "action_receipt" and carry the |
| 219 | + outer envelope's session_id into the yielded record, since it is |
| 220 | + the primary grouping key detectors use.""" |
| 221 | + check = subprocess.run( |
| 222 | + ["pipelock-verify", path, "--key", pubkey_hex], |
| 223 | + capture_output=True, |
| 224 | + text=True, |
| 225 | + ) |
| 226 | + if check.returncode != 0: |
| 227 | + raise RuntimeError( |
| 228 | + f"verification failed: {check.stderr.strip() or check.stdout.strip()}" |
| 229 | + ) |
| 230 | + with open(path) as f: |
| 231 | + for line in f: |
| 232 | + if not line.strip(): |
| 233 | + continue |
| 234 | + entry = json.loads(line) |
| 235 | + if entry.get("type") != "action_receipt": |
| 236 | + continue |
| 237 | + record = entry["detail"]["action_record"] |
| 238 | + record.setdefault("session_id", entry.get("session_id")) |
| 239 | + yield record |
| 240 | + |
| 241 | + |
| 242 | +def default_handler(receipt: dict) -> None: |
| 243 | + """Replace this with your SIEM forwarder, alert pipeline, or |
| 244 | + feature extractor for an LLM classifier.""" |
| 245 | + print( |
| 246 | + f"{receipt['timestamp']} {receipt.get('session_id', '-'):24s} " |
| 247 | + f"{receipt['transport']:20s} {receipt['verdict']:10s} " |
| 248 | + f"{receipt.get('layer', '-'):24s} {receipt.get('pattern', '-')}" |
| 249 | + ) |
| 250 | + |
| 251 | + |
| 252 | +def route(path: str, pubkey_hex: str, handler: Callable[[dict], None]) -> None: |
| 253 | + for receipt in verified_receipts(path, pubkey_hex): |
| 254 | + handler(receipt) |
| 255 | + |
| 256 | + |
| 257 | +if __name__ == "__main__": |
| 258 | + if len(sys.argv) != 3: |
| 259 | + sys.exit("usage: verify_and_route.py <path> <public-key-hex>") |
| 260 | + route(sys.argv[1], sys.argv[2], default_handler) |
| 261 | +``` |
| 262 | + |
| 263 | +Replace `default_handler` with whatever your pipeline needs: |
| 264 | + |
| 265 | +```python |
| 266 | +def route_to_siem(receipt: dict) -> None: |
| 267 | + if receipt["verdict"] == "block": |
| 268 | + forward_to_siem(receipt) |
| 269 | + if receipt["layer"] == "mcp_response_scan": |
| 270 | + score_for_llm_funnel(receipt) |
| 271 | +``` |
| 272 | + |
| 273 | +The shape of the consumer is up to you. The shape of the input is |
| 274 | +fixed by the receipt spec. |
| 275 | + |
| 276 | +## What this does not solve |
| 277 | + |
| 278 | +This is the important section. Signed receipts solve one narrow |
| 279 | +problem. They do not solve several others. |
| 280 | + |
| 281 | +**Compromised mediators can still lie.** A receipt proves Pipelock |
| 282 | +recorded a decision. It does not prove Pipelock made the right |
| 283 | +decision. If a scanner pattern is wrong, the signed record is a |
| 284 | +signed wrong answer. |
| 285 | + |
| 286 | +**Real-time gateways still miss multi-week attacks.** The worked |
| 287 | +example above catches a prompt-injection payload in a single tool |
| 288 | +response because that payload is visible in flight. A slow-boiling |
| 289 | +attack where each individual step looks benign needs a long-window |
| 290 | +detector running on the receipt stream, not a faster gateway. |
| 291 | + |
| 292 | +**Receipts are input to detection, not a substitute for it.** A |
| 293 | +stream of signed records does not tell you which sessions are |
| 294 | +compromised. Someone or something still has to look at the stream |
| 295 | +and make that call. What receipts give you is a trustworthy place |
| 296 | +to look. |
| 297 | + |
| 298 | +**Agent-side attacks are out of scope.** Pipelock sees what the |
| 299 | +agent tries to do on the network. If an attacker has already |
| 300 | +compromised the agent process itself (code execution, same-user |
| 301 | +file access, shared memory), the receipts can document what the |
| 302 | +agent then tried to do, but they cannot prevent the compromise or |
| 303 | +retroactively verify the agent's internal state. |
| 304 | + |
| 305 | +**Same-user deployments have a known ceiling.** If Pipelock runs |
| 306 | +as the same Unix user as the agent, the agent can delete or |
| 307 | +truncate the receipt file. The `demo_capability_separation.py` |
| 308 | +script in the harness demonstrates this limit directly. Running |
| 309 | +Pipelock under a separate user (or in a separate container) is a |
| 310 | +deployment-level fix, not a product-level one. |
| 311 | + |
| 312 | +## Where to go from here |
| 313 | + |
| 314 | +- **Receipt format spec:** <https://pipelab.org/learn/action-receipt-spec/> |
| 315 | +- **Verification mechanics:** [`receipt-verification.md`](receipt-verification.md) |
| 316 | +- **SIEM transport options:** [`siem-integration.md`](siem-integration.md) |
| 317 | +- **Transport coverage matrix:** [`receipt-transports.md`](receipt-transports.md) |
| 318 | +- **Worked example:** [`examples/tool-response-injection/`](/examples/tool-response-injection/) |
| 319 | +- **PyPI verifier:** <https://pypi.org/project/Pipelock-verify/> |
| 320 | + |
| 321 | +If you are integrating Pipelock receipts into a detection pipeline |
| 322 | +and run into something the spec does not cover, open an issue at |
| 323 | +<https://github.com/luckyPipewrench/Pipelock/issues>. |
0 commit comments