[Robustness test] Fix data race by tracking detached watch goroutines #21139
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: henrybear327 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
henrybear327
force-pushed
the
robustness/fix_data_race
branch
from
5186a7d to
355d789
Compare
|
/retest |
Root cause: `RunWatchLoop()` spawned detached goroutines that were not tracked by the main `sync.WaitGroup`. When the parent `SimulateTraffic` is about to exit and calls `wg.Done()`, it will unblocked immediately, allowing Reports() to read from the history while detached goroutines were still writing to it since not all of them have been terminated. Signed-off-by: Chun-Hung Tseng <[email protected]>
henrybear327
force-pushed
the
robustness/fix_data_race
branch
from
355d789 to
5108b71
Compare
|
@henrybear327: The following test failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted filessee 15 files with indirect coverage changes @@ Coverage Diff @@
## main #21139 +/- ##
==========================================
- Coverage 68.46% 68.46% -0.01%
==========================================
Files 429 429
Lines 35242 35242
==========================================
- Hits 24130 24127 -3
- Misses 9723 9728 +5
+ Partials 1389 1387 -2 Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
3 participants
Root cause:
RunWatchLoop()spawned detached goroutines that were not tracked by the mainsync.WaitGroup.When the parent
SimulateTrafficis about to exit and callswg.Done(), it will unblocked immediately, allowing Reports() to read from the history while detached goroutines were still writing to it since not all of them have been terminated.Fixes #21132
Please read https://github.com/etcd-io/etcd/blob/main/CONTRIBUTING.md#contribution-flow.