fix: drain closed accumulator window on re-opened key

[vaibhavtiwari33]

Related issues

Related to #3262

Testing

Long running pipeline with accumulator not returning any messages was allowed to run with a jitter source containing repeated keys. The increase in memory for the accumulator reduced significantly in that scenario

Special notes for reviewers

There still seems to be a memory leak somewhere leading to a slow creeping increase in memory usage, only visible during long runs. Hence, the original issue referenced will be left open.

[codecov]

Codecov Report

❌ Patch coverage is 97.95918% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 83.07%. Comparing base (6e64ec1) to head (ba8fbc6).

Files with missing lines	Patch %	Lines
...c/reduce/reducer/unaligned/windower/accumulator.rs	97.95%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3494      +/-   ##
==========================================
+ Coverage   83.02%   83.07%   +0.05%     
==========================================
  Files         310      310              
  Lines       81426    81426              
==========================================
+ Hits        67605    67646      +41     
+ Misses      13228    13187      -41     
  Partials      593      593              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[tmenjo]

Thank you for a new fix. I've run my pipeline on the Numaflow with this fix.

Screenshots

Memory usage

(Timestamps are in UTC.)

The memory usage of the numa container (yellow line) still went up gradually. However, compared to the chart I had reported in #3461 (comment), its increasing rate looked to get smaller. 16-24 MiB per day by eye. I possibly have a workaround in this rate, such as restarting the Pod for my accumulator every few hundred days to reset the memory usage.

Storage usage

(Timestamps are in +0900.)

There was no problem. It went up and down, but did not increase monotonically.

Logs

(Timestamps are in +0900).

• 0.log.20260708-170939.gz
• 0.log.20260708-174021.gz
• 0.log.20260708-181102.gz
• 0.log.20260708-184153.gz
• 0.log.20260708-191244.gz
• 0.log.20260708-194326.gz
• 0.log.20260708-201407.gz
• 0.log.20260708-204458.gz
• 0.log.20260708-211549.gz
• 0.log.20260708-214620.gz
• 0.log.20260708-221701.gz
• 0.log.20260708-224743.gz
• 0.log.20260708-231824.gz
• 0.log.20260708-234905.gz
• 0.log.20260709-001946.gz
• 0.log.20260709-005027.gz
• 0.log.20260709-012108.gz
• 0.log.20260709-015200.gz
• 0.log.20260709-022241.gz
• 0.log.20260709-025322.gz
• 0.log.20260709-032403.gz
• 0.log.20260709-035454.gz
• 0.log.20260709-042535.gz
• 0.log.20260709-045607.gz
• 0.log.20260709-052648.gz
• 0.log.20260709-055729.gz
• 0.log.20260709-062820.gz
• 0.log.20260709-065901.gz
• 0.log.20260709-073002.gz
• 0.log.20260709-080053.gz
• 0.log.20260709-083125.gz
• 0.log.20260709-090156.gz
• 0.log.20260709-093247.gz
• 0.log.20260709-100328.gz
• 0.log.20260709-103429.gz
• 0.log.20260709-110550.gz
• 0.log.20260709-113732.gz
• 0.log.20260709-121023.gz
• 0.log.20260709-124154.gz
• 0.log.20260709-131215.gz
• 0.log.20260709-134236.gz

Environment

• Numaflow: cbc275a
  • Note that actual commit ID is 78c66f3455afbe6e0508685e01d53bddaf2cf3df because I locally modified and commited image URLs in the spec files to use my own built one. Please be careful when you view debug logs.
• pynumaflow: 0.14.0

[vaibhavtiwari33]

Thanks @tmenjo for trying out this partial fix. I'll open this PR up for review. Meanwhile I'm still investigating the minor leak that even I've observed during my testing, which is now corroborated by you as well.

[vigith]

@vaibhavtiwari33 can you run bytehound and figure out what is happening?