Composite Scenarios Execution - Setup parallely, spam parallely (and in stages) #242

arpitkarnatak · 2025-05-21T17:50:21Z

Motivation

This PR is step 1 for composite scenario execution, as suggested in #145

Solution

Adding a new compose command, similar to docker compose

Composite scenario execution, setup and spam multiple contracts from a single YML file.

Usage: contender compose [OPTIONS]

Options:
  -f, --filename <FILENAME>
          File path for composite scenario file
          
          [default: ./contender-compose.yml]

A contender-compose.yml file looks like

setup:
  simpler:
    testfile: ./scenarios/simpler.toml
    min_balance: 12.1

  uniV2:
    testfile: ./scenarios/uniV2.toml
    rpc_url: http://localhost:8545 # Optional
    min_balance: "11"
    env:
      - Key1=Valu1
      - Key2=Valu2
    private_keys: # Optional (these Private keys are from Anvil)
      - 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80
      - 0x59c6995e998f97a5a0044966f0945389dc9e86dae88c7a8412f4603b6b78690d
    tx_type: eip1559  # Optional

  uniV3:
    testfile: ./scenarios/simple.toml
    rpc_url:  http://localhost:8545 # Optional
    min_balance: 0.01  # Optional
    tx_type: eip1559  # Optional

Currently this only supports setup command, it parses a contender-compose.yml file and deploys multiple contracts from a single contender compose command.

The next step would be to include the spam from this file. Before implementing that, I want to know if all these spams are going to be run parallely and fall under a single run ID, or be separate runs one after the other.

Also, I feel it would be better to have all the logic for the compose file to be a separate crate of its own, similar to testfile. Let me know your thoughts, and any feedback on how to improve/move forward with this.

PR Checklist

Added Tests
Added Documentation
Breaking changes

zeroXbrock · 2025-05-21T23:52:33Z

@arpitkarnatak this is sick, thanks for the PR!

The next step would be to include the spam from this file. Before implementing that, I want to know if all these spams are going to be run parallely and fall under a single run ID, or be separate runs one after the other.

They should all run in parallel with different run IDs. Something I'd like to add in the "compose config" is the ability to define dependencies, so that we may execute them in sequence, but I think the default should be to run them all together at once.

Also, I feel it would be better to have all the logic for the compose file to be a separate crate of its own, similar to testfile. Let me know your thoughts, and any feedback on how to improve/move forward with this.

Yeah I agree, this should live in its own crate.

arpitkarnatak · 2025-05-22T17:20:59Z

@zeroXbrock Thanks for the feedback.

So I'll have to do the following next

Implement spam and it's relevant parameters for running multiple scenarios parallely.
Add a depends_on for setup command so if we have to run certain scenarios sequentially.
Move the logic for compose file into a new crate of its own contender_composefile.
More detailed documentation on contender-compose.yml file specs.

I have one little doubt,

Currently I'm using the pre-existing setup() and spam() functions from inside the compose file. Are we supposed to use that only or take the logic pieces from it into the new compose() function. Pros for using existing functions: Less lines of code, no code repetition , even docker does the same. Cons would be, jumbled logs for all the different scenarios, since each scenario/set of scenarios will be a separate task)

…ion of spam

…e module

arpitkarnatak · 2025-05-26T17:48:13Z

@zeroXbrock So I've done the following

Added spam into compose file, with the stages as mentioned in the requirement. The setups run parallely, and so do the spams for each stage.
contender_composefile is a separate crate now.
There's some documentation for compose file now, highlighting how it maps to the spam and setup commands

Some things that I've had confusions with.

I wanna hold off on implementing depends_on for now, since those dependencies exists in the scenario.toml file itself (as per issue scenario file dependencies #253) so I guess we might wanna be done with scenario file dependencies #253 before this.
There are a few parameters seed, engine_params in SpamCommandArgs and SetupCommandArgs which I don't understand what they mean or wasn't able to make sense of (as in where they come from), so those are hardcoded.

I want to do the following before merge

Write some more tests for the new crate composefile.
Adding additional documentation on each and every parameter of the contender-compose.yml file.

But apart from that, it's ready for review, do let me know what needs fixing/clarity

…te module

…ultiple util files for readability

zeroXbrock · 2025-05-28T20:44:37Z

@arpitkarnatak thanks for the PR! I should have time later in the week to review

zeroXbrock · 2025-07-28T17:45:05Z

sorry for such a long delay @arpitkarnatak -- I've been meaning to get to this but we haven't needed this feature for the tests we're currently doing with contender, so this has fallen by the wayside. I'll take another look and hopefully get this merged soon!

arpitkarnatak · 2025-07-28T18:27:27Z

Absolutely understandable @zeroXbrock please take your time

I was just meaning to get an initial review for the time being to get a sense of whether this is in-line with what you had in mind for this feature, and I could polish it further if it was. Thanks and would be looking forward to the review and comments

zeroXbrock

thank you for being so patient @arpitkarnatak. I think we can get this merged soon!

Just had a few comments. Let me know if I can clarify anything!

zeroXbrock · 2025-05-21T23:55:44Z

contender-compose.yml

+    private_keys: # Optional (these Private keys are from Anvil)
+      - 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80
+      - 0x59c6995e998f97a5a0044966f0945389dc9e86dae88c7a8412f4603b6b78690d


private keys should only be specified as a cli param -- we don't want people accidentally saving their keys into files that they might share with the world

Good idea, I think it would be best that way, will change it accordingly

zeroXbrock · 2025-05-21T23:56:12Z

crates/cli/Cargo.toml

 tracing = { workspace = true }
 tracing-subscriber = { workspace = true }
 webbrowser = { workspace = true }
+yaml-rust2 = "0.10.2"


I know I specified yaml when I wrote the issue but I wonder if toml would be better -- wdyt?

Toml can be nice, since we're using it for the other configurations too. For this case, I would prefer YAML solely because of the readability and syntax familiarity with Docker compose syntax.

zeroXbrock · 2025-08-29T21:28:52Z

crates/cli/src/commands/setup.rs

+            private_keys: setup_object.private_keys,
+            env: setup_object.env,
+            min_balance: setup_object.min_balance,
+            tx_type: if setup_object.tx_type == *"eip1559" {


should use enum matching here instead of string comparison

zeroXbrock · 2025-08-29T21:31:14Z

crates/cli/src/commands/spam.rs

+            },
+            loops: spam_object.loops,
+
+            // TODO: Need more knowledge, hence hardcoding them for now


these should inherit their values from the same CLI args that we use to run spam.
They're just CLI args, but you'll have to check the spam impl to see how the defaults are filled in.

zeroXbrock · 2025-08-29T21:31:54Z

crates/composefile/src/types.rs

+    pub timeout_secs: u64,
+    pub env: Option<Vec<(String, String)>>,
+    pub loops: Option<u64>,
+    // TODO: These params are hardcoded for now, I'll need some more info in these before implementing them


see above comment about CLI args inheritance

zeroXbrock · 2025-08-29T21:32:34Z

crates/composefile/src/utils/get_tx_type.rs

+use std::error::Error;
+use yaml_rust2::Yaml;
+
+pub fn get_tx_type(yaml_object: &LinkedHashMap<Yaml, Yaml>) -> Result<String, Box<dyn Error>> {


we should delete this and use enum matching instead

This utility is for handling strings from Compose file. These functions are to extract strings from YAML

In the compose file crate, we have utility to read YAML files, and convert each setup and spam scenario into a JSON, and then from the command we convert that JSON to SetupCommandArgs and SpamCommandArgs (in crates/cli/src/commands/setup and crates/cli/src/commands/spam)

I feel like there's a simpler approach. There's too much boilerplate here (and in the other utils / yaml parsing functions in general), and the tests are essentially just testing the YAML parser.

serde_yaml exists, which would parse all of this automatically, but it's not maintained anymore and probably isn't compatible with the modern version of serde that we already use. I'd prefer to swap out YAML for TOML, so we can use serde.

zeroXbrock · 2025-09-09T23:39:43Z

hey @arpitkarnatak just wondering if you're still working on this

arpitkarnatak · 2025-09-11T05:56:08Z

@zeroXbrock Yes I am on this, I'll try to get this done over the weekend with the changes you asked

…posefile

arpitkarnatak · 2025-09-14T16:20:57Z

Hello @zeroXbrock I added the following features

Private keys via CLI arg
Adding Engine Params from compose file

The relevant snippets are in the last 4 commits

zeroXbrock

nice work so far @arpitkarnatak -- thanks so much! The system works well, but I left a few suggestions for the code in the comments below.

My biggest concern is that the manual YAML parsing adds too much boilerplate code to maintain efficiently. I think TOML is the way to go -- it'll allow us to use serde (which we already use) and reduce the amount of code needed by a huge amount.

Also, the version of contender your branch is based off is pretty old -- you may want to merge main into your branch soon, but it's up to you whether you do that before or after making the changes I suggested. I imagine it will be easier to do afterwards, since there should be less code to change once the duplication is removed.

zeroXbrock · 2025-09-16T21:07:03Z

contender-compose.yml

Not sure this should be in the project root -- maybe it would fit better under scenarios/? Or we could have a new directory called composite-scenarios/ or something like that.
We should also use valid values, so that users with a simple node (e.g. anvil w/ default config) running on localhost:8545 can run the compose file without error. Any advanced examples we want to show off should go in a document under docs/.

maybe we should also enforce a .compose.toml filename restriction, so users don't accidentally try to run core scenario files as compose files

zeroXbrock · 2025-09-16T22:15:50Z

contender-compose.yml

@@ -0,0 +1,48 @@
+setup:


Would be cool if we didn't have to manually call setup. Feels like manually specifying rpc_urls for setup could get confusing in a large file (i.e. the user tries to spam on an alternate chain but forgets to run setup on that chain).

Instead, we could check if the scenario TOML file has any setup directives, then run TestScenario::estimate_setup_cost to find the required min_balance, then check the spam directives in the compose file to see which chains need to run which setup and run it automatically at the start.

zeroXbrock · 2025-09-16T22:18:09Z

crates/cli/src/commands/composite.rs

+
+    futures::future::join_all(setup_tasks).await;
+
+    info!("================================================================================================= Done Composite run for setup =================================================================================================");


let's remove the equal signs from this -- simple logs are fine

zeroXbrock · 2025-09-16T22:25:06Z

crates/composefile/src/types.rs

+pub struct SpamCommandArgsJsonAdapter {
+    pub scenario: String,
+    pub rpc_url: String,
+    pub builder_url: Option<String>,
+    pub txs_per_block: Option<u64>,
+    pub txs_per_second: Option<u64>,
+    pub duration: u64,
+    pub private_keys: Option<Vec<String>>,
+    pub min_balance: String,
+    pub tx_type: String,
+    pub timeout_secs: u64,
+    pub env: Option<Vec<(String, String)>>,
+    pub loops: Option<u64>,
+    pub seed: Option<String>,
+
+    pub call_fcu: bool,
+    pub use_op: bool,
+    pub auth_rpc_url: Option<String>,
+    pub jwt_secret: Option<String>,
+    pub disable_reporting: bool,
+    pub gas_price_percent_add: Option<u64>,
+}
+
+// In `contender_cli` we can retrieve the SetupCommandArgs struct from this using SetupCommandArgs::from<SetupCommandArgsJsonAdapter>()
+#[derive(Deserialize, Debug, Clone)]
+pub struct SetupCommandArgsJsonAdapter {
+    pub testfile: String,
+    pub rpc_url: String,
+    pub private_keys: Option<Vec<String>>,
+    pub min_balance: String,
+    pub tx_type: String,
+    pub env: Option<Vec<(String, String)>>,
+    pub call_fcu: bool,
+    pub use_op: bool,
+    pub auth_rpc_url: Option<String>,
+    pub jwt_secret: Option<String>,
+    pub seed: Option<String>,
+    // pub engine_params: EngineParams,
+}


These include lots of duplicate variables from structs that already exist in contender_cli.
Since compose is a cli command, I think it would make more sense to just move this whole crate into a module in crates/cli/src/commands/.

Then, instead of duplicating the variables manually for this struct, just compose these structs using the existing structs and #[serde(flatten)]

zeroXbrock · 2025-09-16T22:40:22Z

crates/composefile/src/utils/get_tx_type.rs

+use std::error::Error;
+use yaml_rust2::Yaml;
+
+pub fn get_tx_type(yaml_object: &LinkedHashMap<Yaml, Yaml>) -> Result<String, Box<dyn Error>> {


I feel like there's a simpler approach. There's too much boilerplate here (and in the other utils / yaml parsing functions in general), and the tests are essentially just testing the YAML parser.

serde_yaml exists, which would parse all of this automatically, but it's not maintained anymore and probably isn't compatible with the modern version of serde that we already use. I'd prefer to swap out YAML for TOML, so we can use serde.

zeroXbrock · 2025-10-15T21:38:04Z

hey @arpitkarnatak are you still working on this?

arpitkarnatak · 2025-10-22T17:35:13Z

Hey @zeroXbrock I can't give more time to this in near future at least given my current commitments. Sorry :(

zeroXbrock · 2025-10-23T19:10:56Z

Hey @zeroXbrock I can't give more time to this in near future at least given my current commitments. Sorry :(

no worries, thanks for letting me know!

arpitkarnatak added 2 commits May 21, 2025 23:08

added composite scenarios - setting up multiple contracts

9d7a6bf

Added better documentation for compose command

171b220

arpitkarnatak requested a review from zeroXbrock as a code owner May 21, 2025 17:50

arpitkarnatak and others added 2 commits May 21, 2025 23:23

Merge branch 'main' into feat/composite-scenaris

7c91fdf

Clippy fixes

bb62181

arpitkarnatak added 6 commits May 23, 2025 20:21

Run setup for contracts parallely

4b15167

Added parsing for spam commands with multiple stages, parallel execut…

cdf64f4

…ion of spam

Added new struct ComposeFile, moved compose file parsing into separat…

34c9dd3

…e module

Added separate composefile container

e4c262d

Added readme for contender compose file

52ba64c

added deleted parameters for setup return type in composefile

2a437b2

arpitkarnatak changed the title ~~Composite Scenarios Execution (Part 1) - Running setup for multiple contracts from a single contender-compose.yml file,~~ Composite Scenarios Execution - Setup parallely, spam parallely (and in stages) May 26, 2025

arpitkarnatak added 3 commits May 27, 2025 23:30

Changed composefile struct properties, moved util functions in separa…

b40f725

…te module

Removed unwrap statements in multithreaded spam calls

92e08ec

Test coverage for composefile, reducing filesizes by splitting into m…

6f9e430

…ultiple util files for readability

Add readme for writing composefiles

d264c62

zeroXbrock reviewed Aug 29, 2025

View reviewed changes

arpitkarnatak added 3 commits September 14, 2025 11:25

Pattern matching for tx types in setup and spam commands

bfbd5ea

Added parameter -p to add private keys, removed private keys from com…

7629a88

…posefile

Added method to have engine params in composefile

ad913cc

Removed from method

dd613f6

zeroXbrock reviewed Sep 16, 2025

View reviewed changes

zeroXbrock mentioned this pull request Dec 3, 2025

composite scenario execution #145

Open


		futures::future::join_all(setup_tasks).await;

		info!("================================================================================================= Done Composite run for setup =================================================================================================");

Composite Scenarios Execution - Setup parallely, spam parallely (and in stages) #242

Are you sure you want to change the base?

Composite Scenarios Execution - Setup parallely, spam parallely (and in stages) #242

Uh oh!

Conversation

arpitkarnatak commented May 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Solution

PR Checklist

Uh oh!

zeroXbrock commented May 21, 2025

Uh oh!

arpitkarnatak commented May 22, 2025

Uh oh!

arpitkarnatak commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zeroXbrock commented May 28, 2025

Uh oh!

zeroXbrock commented Jul 28, 2025

Uh oh!

arpitkarnatak commented Jul 28, 2025

Uh oh!

zeroXbrock left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zeroXbrock Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zeroXbrock commented Sep 9, 2025

Uh oh!

arpitkarnatak commented Sep 11, 2025

Uh oh!

arpitkarnatak commented Sep 14, 2025

Uh oh!

zeroXbrock left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zeroXbrock Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zeroXbrock commented Oct 15, 2025

Uh oh!

arpitkarnatak commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zeroXbrock commented Oct 23, 2025

Uh oh!

arpitkarnatak commented May 21, 2025 •

edited

Loading

arpitkarnatak commented May 26, 2025 •

edited

Loading

zeroXbrock Sep 16, 2025 •

edited

Loading

zeroXbrock left a comment •

edited

Loading

zeroXbrock Sep 16, 2025 •

edited

Loading

arpitkarnatak commented Oct 22, 2025 •

edited

Loading