Skip to content

fix: prevent stale reads in MCP server with concurrent processes#66

Open
jeeshofone wants to merge 3 commits intosirsjg:mainfrom
CompactSaaS-Org:fix/sqlite-race-condition
Open

fix: prevent stale reads in MCP server with concurrent processes#66
jeeshofone wants to merge 3 commits intosirsjg:mainfrom
CompactSaaS-Org:fix/sqlite-race-condition

Conversation

@jeeshofone
Copy link
Contributor

Problem

Agents using MCP via docker exec cannot see each other's changes because each process reads the database once at startup and never refreshes.

Example:

  • Agent 1 lists tasks → sees 102 tasks
  • Agent 2 creates task → writes to DB
  • Agent 1 lists tasks → still sees 102 tasks ❌

Solution

Add adapter.read() before each MCP tool call to refresh data from disk.

server.setRequestHandler(CallToolRequestSchema, async (request) => {
  // Refresh data from disk before each operation
  if (!isServerMode()) {
    const adapter = getStorageAdapter();
    adapter.read();
  }
  // ... handle tool call
});

Testing

Automated Test

# Run cross-agent visibility test
bun tests/concurrency/test-cross-agent-visibility.ts

Manual Test

# Agent 1: Initial count
echo '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"list_tasks","arguments":{"project_id":"0gcsjpp"}}}' | docker exec -i flux-web bun packages/mcp/dist/index.js 2>/dev/null | tail -1 | jq '.result.content[0].text | fromjson | length'
# Output: 102

# Agent 2: Create task
echo '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"create_task","arguments":{"project_id":"0gcsjpp","title":"Test"}}}' | docker exec -i flux-web bun packages/mcp/dist/index.js

# Agent 1: Final count (should be 103)
echo '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"list_tasks","arguments":{"project_id":"0gcsjpp"}}}' | docker exec -i flux-web bun packages/mcp/dist/index.js 2>/dev/null | tail -1 | jq '.result.content[0].text | fromjson | length'
# Output: 103 ✅

Test Results

[Agent 1] Initial task count: 102
[Agent 2] Created task "Test task 1769629970" with ID: 9hbg49m
[Agent 1] Final task count: 103
✅ SUCCESS: Agent 1 sees Agent 2's task immediately!

Changes

  • packages/mcp/src/index.ts: Add adapter.read() before each tool call
  • tests/concurrency/: Add comprehensive test suite
    • test-stale-reads.ts: Demonstrates the issue
    • test-cross-agent-visibility.ts: Verifies the fix
    • README.md: Documentation

Breaking Changes

None. Backward compatible.

Related

This PR completes the concurrency fix suite for multi-agent workflows.

Fixes two critical concurrency issues when multiple agents use MCP via docker exec:

1. Race condition on writes (lost updates)
   - Multiple processes overwriting each other's changes
   - Fixed with transaction-based merge in SQLite adapter
   - Test: 30 concurrent writes → 0 lost (was 29 lost, 97% loss rate)

2. Stale reads (agents not seeing each other's changes)
   - Each MCP process reads DB once at startup, never refreshes
   - Fixed by calling adapter.read() before each MCP tool call
   - Test: Agent 1 immediately sees Agent 2's new task

Changes:
- SQLite adapter: Add transaction-based write with mergeById helper
- MCP server: Refresh data from disk before each operation
- Tests: Comprehensive unit tests + reproduction scripts
- Docker: Switch to SQLite by default in docker-compose.yml

Fixes sirsjg#60
Keep additional test files and organization improvements
@jeeshofone jeeshofone requested a review from sirsjg as a code owner January 28, 2026 19:59
Upstream added blob storage which requires FLUX_DIR to locate the .flux directory.
Without it, server tries to create /home/flux/.flux/blobs and crashes with permission denied.

Changes:
- Add FLUX_DIR=/app/packages/data to docker-compose.yml
- Single volume mount (./data:/app/packages/data) is sufficient
- Blobs directory auto-created by mkdirSync in createFilesystemBlobStorage

Tested:
- Clean install (no blobs dir) ✅
- Stale read fix still works ✅
- Write race condition fix still works ✅
@jeeshofone jeeshofone force-pushed the fix/sqlite-race-condition branch from 10c6795 to 85bd6ea Compare January 28, 2026 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stale Reads: Agents Can't See Each Other's Changes

1 participant