Skip to content

PVC protection unit tests flake #1376

@jsafrane

Description

@jsafrane

What happened:
Random unit tests flake with Claim fake-pvc does not have clone protection finalizer set. The failure rate is very small, "4989 runs so far, 3 failures (0.06%)". And it's always a different test.
Example from our CI.

How to reproduce it:
With v5.2.0 tag:

$ # install `stress` tool
$ go install golang.org/x/tools/cmd/stress@latest

$ # Compile the unit tests
$ git checkout v5.2.0
$ cd pkg/controller
$ go test -c -race

$ stress -p 100 ./controller.test

Where 100 is nr. of tests to run in parallel. Using ~ 5x nr. of my CPUs (incl. hyperthreading ones), still got CPUs ~50% idle. I got 5 flakes in 30 minutes.

Individual failures:

--- FAIL: TestProvisionFromPVC (7.15s)
    --- FAIL: TestProvisionFromPVC/provision_with_pvc_data_source_different_storage_classes_when_CrossNamespaceVolumeDataSource_feature_enabled (0.11s)
        controller_test.go:6690: Claim fake-pvc does not have clone protection finalizer set
--- FAIL: TestProvisionFromPVC (7.15s)
    --- FAIL: TestProvisionFromPVC/provision_nil_mode_data_source_is_nil (0.11s)
        controller_test.go:6690: Claim fake-pvc does not have clone protection finalizer set
--- FAIL: TestProvisionFromPVC (7.17s)
    --- FAIL: TestProvisionFromPVC/provision_with_pvc_data_source_different_storage_classes_when_CrossNamespaceVolumeDataSource_feature_enabled (0.11s)
        controller_test.go:6690: Claim fake-pvc does not have clone protection finalizer set
--- FAIL: TestProvisionFromPVC (7.04s)
    --- FAIL: TestProvisionFromPVC/provision_with_pvc_data_source_destination_too_small (0.12s)
        controller_test.go:6690: Claim fake-pvc does not have clone protection finalizer set
--- FAIL: TestProvisionFromPVC (7.16s)
    --- FAIL: TestProvisionFromPVC/provision_block_data_source_is_block (0.12s)
        controller_test.go:6690: Claim block-pv does not have clone protection finalizer set

Metadata

Metadata

Labels

good first issueDenotes an issue ready for a new contributor, according to the "help wanted" guidelines.help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.kind/flakeCategorizes issue or PR as related to a flaky test.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions