Skip to content

Fix 1097: Nested Tests and Python Dependency Loading#1107

Open
isc-dchui wants to merge 3 commits intomainfrom
fix-1097
Open

Fix 1097: Nested Tests and Python Dependency Loading#1107
isc-dchui wants to merge 3 commits intomainfrom
fix-1097

Conversation

@isc-dchui
Copy link
Copy Markdown
Collaborator

Description

  • Fixes Python wheel test failure + silent CI failure #1097
  • Two part fix:
    • Fixing the suppression of test failures.
      • This stems from nested test/verify calls, e.g. in the scopes integration test, we call scope-test verify -only, so when zpm verify -only is called, we end up nesting.
      • The test resource processor wasn't built to handle nesting due to its use of a single process-private global ^||%UnitTest.Manager.LastResult to track test runs. This is killed at the start of the running of tests, which causes problems with nesting: the parent's results are killed by the child
      • The fix is to instead use a different process-private global ^||%UnitTest.Manager.AllResultsCount that collates individual instantiations.
      • There are two integration test classes that use nested tests: Scopes and FileCopy.
      • Since this bug is dependent on test ordering, i.e. will only occur if a failing test is before a nested test, it didn't occur previously.
      • Note: ^||%UnitTest.Manager.LastResult has been completely removed since IPM will no longer use it, so any custom code built on top of this will break
    • Fixing the Python dependencies from requirements.txt not correctly overriding the Python wheels.
      • FAILED Test.PM.Integration.ProcessPythonWheel:TestWheelAndReqsPresentOnline: AssertEquals - reqVer== "1.6.4" was '1.6.2'.
      • It appears that the problem isn't that 1.6.4 was not installed, but that both 1.6.2 and 1.6.4 were installed and Python was selecting the wrong one.
      • Solution is to cleanup the other installation when a different version is specified in requirements.txt.
      • A bit unclear to me why this has only cropped up recently and not earlier; maybe some change in the environment?

Testing

Ran zpm verify -only and the tests pass. Also made sure if there is a failing test, the verify phase will correctly fail. For nested tests, made sure that if the nested test fails, it will be properly reported in the summary at the end.

Checklist

  • This branch has the latest changes from the main branch rebased or merged.
  • Changelog entry added.
  • Unit (zpm test -only) and integration tests (zpm verify -only) pass.
  • Style matches the style guide in the contributing guide.
  • Documentation has been/will be updated
    • Source controlled docs, e.g. README.md, should be included in this PR and Wiki changes should be made after this PR is merged (add an extra issue for this if needed)
  • Pull request correctly renders in the "Preview" tab.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python wheel test failure + silent CI failure

1 participant