Skip to content

fix(test): gremlin restarts all instances of a node together#2245

Open
beskrovnykh wants to merge 1 commit intoton-blockchain:testnetfrom
beskrovnykh:fix/test-consensus-gremlin-double-node
Open

fix(test): gremlin restarts all instances of a node together#2245
beskrovnykh wants to merge 1 commit intoton-blockchain:testnetfrom
beskrovnykh:fix/test-consensus-gremlin-double-node

Conversation

@beskrovnykh
Copy link
Copy Markdown

Summary

Gremlin previously picked a single instance to kill. For double-node validators (two instances sharing the same key), this left the other instance running. The restarted instance would vote differently, triggering CHECK "We produced conflicting votes!" (pool.cpp:532).

Now gremlin selects a node and restarts all its instances together, so double-nodes stay in sync after a gremlin cycle.

Reproduction (before fix)

./test-consensus -n 8 --n-double-nodes 2 --gremlin-period 0.5:2 \
  --gremlin-downtime 0.3:1.0 --gremlin-n 2 --net-ping 0.05:0.2 \
  --duration 30
# CHECK crash 3/3 runs

Test plan

  • ./test-consensus -n 8 --gremlin-period 5:10 --gremlin-downtime 2:4 --duration 30 — gremlin without double-nodes still works
  • ./test-consensus -n 8 --n-double-nodes 2 --gremlin-period 0.5:2 --gremlin-downtime 0.3:1.0 --gremlin-n 2 --net-ping 0.05:0.2 --duration 30 — no longer crashes

Previously gremlin picked a single instance to kill. For double-node
validators (two instances sharing the same key), this left the other
instance running. The restarted instance would vote differently,
triggering CHECK "We produced conflicting votes!" (pool.cpp:532).

Now gremlin selects a node and restarts all its instances together,
so double-nodes stay in sync after a gremlin cycle.
@beskrovnykh beskrovnykh force-pushed the fix/test-consensus-gremlin-double-node branch from 825110c to aba49a5 Compare March 29, 2026 16:24
@DanShaders
Copy link
Copy Markdown
Collaborator

I'm not entirely convinced I want to keep test-consensus as it is currently written at all, so this will wait a better time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants