Fix precompile flaky tests

[kipawaa]

Context:
precompiled decomposition rules tests were failing flaky tests because the decomposition rules were being added to the graph multiple times, causing warnings other than those expected.

Description of the Change:
Ensures the tests and their iterations are independent via the local_decomps context manager from PL.

Benefits:
Improved pytest infrastructure, fixes precompile decomposition rules flaky failures.

Possible Drawbacks:

Related GitHub Issues:

[dime10]

Is the flaky testing itself that was causing the rules to be created multiple times, or did it just reveal that this was possible?

[kipawaa]

Is the flaky testing itself that was causing the rules to be created multiple times, or did it just reveal that this was possible?

@dime10 The PL graph is global and hence persistent across multiple runs of the test, so the issue is the flaky testing itself. I discussed with @paul0403 and we agreed that these tests likely don't need to be tested for flakiness. A part of this decision was also that the only way to remove rules from the graph is to use PL internals (see PL tests).

[dime10]

@dime10 The PL graph is global and hence persistent across multiple runs of the test, so the issue is the flaky testing itself. I discussed with @paul0403 and we agreed that these tests likely don't need to be tested for flakiness. A part of this decision was also that the only way to remove rules from the graph is to use PL internals (see PL tests).

I see, so the issue is that we're adding the same op to the graph database multiple times, which is not our intention. I think it is okay to avoid flaky testing then.

However, separate from the flaky testing is the issue of global state changes. That is a big concern in tests because it breaks the independence assumption! Tests can affect each other and this effect can be sensitive to test ordering, or whether a given test fails or succeeds, causing difficult to reproduce issues. We've dealt with this a couple of times already when global state was involved (e.g. capture on/off), so I would say it's imperative to figure out a way to test these features without mutating (or at least restoring) global state.

[kipawaa]

However, separate from the flaky testing is the issue of global state changes.

Thanks @dime10 that's a great point, I'll look into how PL manages the global state and apply a better solution!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.98%. Comparing base (5dd28aa) to head (3a81e46).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2761      +/-   ##
==========================================
- Coverage   96.99%   96.98%   -0.02%     
==========================================
  Files         165      165              
  Lines       18460    18460              
  Branches     1783     1783              
==========================================
- Hits        17906    17904       -2     
- Misses        398      399       +1     
- Partials      156      157       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.