Cache miss on engine_newPayload to builder causes slow payload validation

[sieniven]

Issue

When op-rbuilder sets state root calculation to be false (--flashblocks.disable-state-root), it is observed that the builder builds and returns the payload without a stateroot. However because the executed block state cache has no state root, this results in a different block hash being computed.

Since external state root calculation is shifted to rollup boost, it sends an FCU to the default execution client to pre-warm the Reth client, and also calculates the state root on engine_getPayload. However, this results in the block hash being different from what is on the op-rbuilder, and this execution payload is subsequently sent back to the CL.

However, on engine_newPayload (op-seq's PayloadProcessEvent trigger), because the payload now contains a stateroot, this results in a cache miss on engine_newPayload API call in the engine state's the pending block. Even though the payload is built locally, op-rbuilder goes through the lengthy payload validation (re-execution of the transactions and computing the state root) and occupies sequencing protocol time, causing the TPS of the chain to drop significantly.

2025-11-25T08:42:44.640084Z  INFO Flashblock built flashblock_index=4 current_gas=119917267 current_da=342100 target_flashblocks=5
2025-11-25T08:42:44.640917Z  INFO Block added to canonical chain number=8596245 hash=0x3ce3583e4730e553ba6a324c43ea8a39e217fce90d482d4ec65cf90eba95c1c4 peers=5 txs=3422 gas_used=119.92Mgas gas_throughput=11456.70Ggas/second gas_limit=150.00Mgas full=79.9% base_fee=9.96Gwei blobs=0 excess_blobs=0 elapsed=10.467µs timing=Produce[Insert[validateExec<0ns>, insertTree<4µs>, total<4µs>]]
2025-11-25T08:42:44.730345Z  INFO Building flashblock block_number=8596245 flashblock_index=5 target_gas=150000000 gas_used=119917267 da_used=342100 block_gas_used=150000000
2025-11-25T08:42:44.853123Z  INFO Flashblock built flashblock_index=5 current_gas=149910651 current_da=427700 target_flashblocks=5
2025-11-25T08:42:44.853888Z  INFO Block added to canonical chain number=8596245 hash=0x219d0ebb543d779b6977dccb59f550157c5ef6e570789b52ee3513fb3d4fb8f0 peers=5 txs=4278 gas_used=149.91Mgas gas_throughput=18755.24Ggas/second gas_limit=150.00Mgas full=99.9% base_fee=9.96Gwei blobs=0 excess_blobs=0 elapsed=7.993µs timing=Produce[Insert[validateExec<0ns>, insertTree<3µs>, total<3µs>]]
2025-11-25T08:42:44.955469Z  INFO Resolve kind Earliest
2025-11-25T08:42:44.960116Z  INFO Block added to canonical chain number=8596245 hash=0x219d0ebb543d779b6977dccb59f550157c5ef6e570789b52ee3513fb3d4fb8f0 peers=5 txs=4278 gas_used=149.91Mgas gas_throughput=25026.82Ggas/second gas_limit=150.00Mgas full=99.9% base_fee=9.96Gwei blobs=0 excess_blobs=0 elapsed=5.99µs timing=Produce[Insert[validateExec<0ns>, insertTree<1µs>, total<1µs>]]
2025-11-25T08:42:45.435089Z  INFO Received block from consensus engine number=8596245 hash=0x83abb38ffb242d36846889c8f6339845a81b628e0341740f742befaa3023b0da
2025-11-25T08:42:46.596769Z  INFO State root task finished state_root=0x976fddf73f5af6002412db684b9ddbfa01e917b6e9ecfdc9dbdef44ba07dd002 elapsed=174.378931ms
2025-11-25T08:42:46.597860Z  INFO Block added to canonical chain number=8596245 hash=0x83abb38ffb242d36846889c8f6339845a81b628e0341740f742befaa3023b0da peers=5 txs=4278 gas_used=149.91Mgas gas_throughput=128.94Mgas/second gas_limit=150.00Mgas full=99.9% base_fee=9.96Gwei blobs=0 excess_blobs=0 elapsed=1.162657605s timing=Produce[Insert[validateExec<1162ms>, insertTree<3µs>, total<1162ms>]]
2025-11-25T08:42:46.601590Z  INFO Canonical chain committed number=8596245 hash=0x83abb38ffb242d36846889c8f6339845a81b628e0341740f742befaa3023b0da elapsed=3.593355ms
2025-11-25T08:42:46.601995Z  INFO Spawn block building job