|
1 | | -This directory enables integration of Antithesis with etcd. There are 4 containers running in this system: 3 that make up an etcd cluster (etcd0, etcd1, etcd2) and one that "[makes the system go](https://antithesis.com/docs/getting_started/basic_test_hookup/)" (client). |
| 1 | +# etcd Antithesis tests |
| 2 | + |
| 3 | +This document describes the etcd test integration with [Antithesis]. |
| 4 | +Antithesis provides a testing platform that allows you to explore edge cases, race conditions, and rare |
| 5 | +bugs that are difficult or impossible to reproduce in a normal environment. |
| 6 | + |
| 7 | +[Antithesis]: https://antithesis.com/ |
| 8 | + |
| 9 | +## Robustness vs Antithesis tests |
| 10 | + |
| 11 | +[Antithesis] runs the robustness tests inside their |
| 12 | +[deterministic simulation testing](https://antithesis.com/resources/deterministic_simulation_testing/) |
| 13 | +environment and [fault injection](https://antithesis.com/docs/environment/fault_injection/). |
| 14 | + |
| 15 | +For more details on robustness tests, see the [robustness directory](../robustness). |
| 16 | + |
| 17 | +## Antithesis Setup |
| 18 | + |
| 19 | +The setup consists of a 3-node etcd cluster and a client container, orchestrated |
| 20 | +via [Docker Compose](https://antithesis.com/docs/getting_started/setup/). |
| 21 | + |
| 22 | +During the etcd Antithesis test suite the etcd server is built with the following patches: |
| 23 | + |
| 24 | +* **Critical code locations**: We replace etcd `gofail` comments (which signify |
| 25 | + code locations important for failure injection in robustness tests) with |
| 26 | + Antithesis `assert.Reachable`. This guides Antithesis to explore the |
| 27 | + execution space around these points. |
| 28 | +* **Assertions**: We change etcd `verify` package assertions to Antithesis |
| 29 | + `assert.Always`, encouraging the platform to try and break those assertions. |
| 30 | +* **Instrumentation**: The etcd binary is instrumented using |
| 31 | + `antithesis-go-instrumentor` to enable coverage tracking and feedback for |
| 32 | + the Antithesis platform. |
| 33 | + |
| 34 | +The Antithesis etcd tests configure the |
| 35 | +[Test Composer](https://antithesis.com/docs/test_templates/test_composer_reference/) |
| 36 | +in the following way: |
| 37 | + |
| 38 | +* **`entrypoint`**: |
| 39 | + * Waits for all etcd nodes to be healthy. |
| 40 | + * Emits the `setup_complete` message to Antithesis to start the testing phase. |
| 41 | +* **`singleton_driver_traffic`**: |
| 42 | + * Generates robustness test traffic against the cluster while faults are injected. |
| 43 | + * Runs as a [Singleton Driver Command], meaning it is the only one generating traffic. |
| 44 | + * All generated traffic is saved as an operation history and stored on a shared volume. |
| 45 | +* **`finally_validation`**: |
| 46 | + * Runs as a [Finally Command], meaning it is the last to run, with failure injection disabled. |
| 47 | + * Reads the history of operations and validates them using the robustness test validation logic. |
| 48 | + * Results of robustness tests are executed as Antithesis `assert.Always` assertions. |
| 49 | + * Similar to robustness tests, it emits a visualization of the operations |
| 50 | + history to an HTML file that is uploaded to the Antithesis platform. |
| 51 | + |
| 52 | +[Singleton Driver Command]: https://antithesis.com/docs/test_templates/test_composer_reference/#singleton-driver |
| 53 | +[Finally Command]: https://antithesis.com/docs/test_templates/test_composer_reference/#finally-command |
2 | 54 |
|
3 | 55 | # Running tests with docker compose |
4 | 56 |
|
|
0 commit comments