Skip to content

A bunch of CTracer selection (and other) tests fail on Fedora-based dev envs #2050

@webknjaz

Description

@webknjaz

Describe the bug

So while rebasing #1952, @OlenaYefymenko started seeing an unexpected test failure. It turned out to be a test added in bea6204tests/test_process.py::CoverageCoreTest::test_core_request_ctrace_but_missing.

Her dev env is Fedora. It doesn't fail on my Gentoo Linux laptop. So I tried reproducing that in a Fedora container and ended up discovering several things.

  1. The contributing doc does not explicitly list OS-level prerequisites (gcc, for example) — so a less experienced contributor might be greeted with a wall of errors on the very first tox run. This would be good to improve: [docs] List platform-level prerequisites in the contributing doc #2051.
  2. There's four tests failing in that env. I haven't checked whether this is due to inadequate test isolation or there's real bugs in there. I think this calls for adding an additional job to the CI running a Fedora container (maybe some more too).

To Reproduce

Here's a one-liner for running what I ran in an F43 container:

$ podman run --rm -it fedora:43 bash -x -c 'dnf install -y gcc git python3.13-freethreading python3.14-freethreading tox && git clone https://github.com/nedbat/coveragepy.git && cd coveragepy && tox'

GHA-based repro: #2053 / https://github.com/nedbat/coveragepy/actions/runs/17766812690/job/50492157797?pr=2053#step:7:46.

Expected behavior

All the tests should pass.

Additional context

The entire log is big but here's one of the tox envs that has 4 failing tests (others have fewer failures):

py39: commands[2]> python igor.py test_with_core pytrace
=== CPython 3.9.23 (main Jul 25 2025 00:00:00) (gil) with Python tracer (/usr) ===
bringing up nodes...
.................................................................................................................................................... [  9%]
...............x....s....s......s.....ss................s.s.......................................s................................................. [ 19%]
.........s..sss.....ss.........s..s................................................................................................................. [ 29%]
.................................................................................................................................................... [ 39%]
.................................................................................................................................................... [ 49%]
......................................................................................................s..........s.................................. [ 58%]
.............................................................................................................ss.s................................... [ 68%]
...................s.....ss.s...........................ss.......................sssss.s..ssssssssss.ssssss........s.......s.........s.............. [ 78%]
.............s.s.s..............s..............s.F..s...................................F........................................................... [ 88%]
................................s............................................................................................s...................... [ 98%]
.......................ss..FF                                                                                                                        [100%]
========================================================================= FAILURES =========================================================================
____________________________________________________________ CoverageCoreTest.test_core_default ____________________________________________________________
[gw7] linux -- Python 3.9.23 /coveragepy/.tox/py39/bin/python

self = <tests.test_process.CoverageCoreTest object at 0x7f4b4187c880>

    def test_core_default(self) -> None:
        self.del_environ("COVERAGE_CORE")
        self.make_file("numbers.py", "print(123, 456)")
        out = self.run_command("coverage run --debug=sys numbers.py")
        assert out.endswith("123 456\n")
        core = re_line(r" core:", out).strip()
        warns = re_lines(r"\(no-ctracer\)", out)
        if env.SYSMON_DEFAULT:
            assert core == "core: SysMonitor"
            assert not warns
        elif self.has_ctracer:
            assert core == "core: CTracer"
            assert not warns
        else:
>           assert core == "core: PyTracer"
E           AssertionError: assert 'core: CTracer' == 'core: PyTracer'
E             
E             - core: PyTracer
E             ?       ^^
E             + core: CTracer
E             ?       ^

/coveragepy/tests/test_process.py:1343: AssertionError
------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------
-- sys -------------------------------------------------------
               coverage_version: 7.10.7a0.dev1
                coverage_module: /coveragepy/.tox/py39/lib64/python3.9/site-packages/coverage/__init__.py
                           core: CTracer
                        CTracer: available from /coveragepy/.tox/py39/lib64/python3.9/site-packages/coverage/tracer.cpython-39-x86_64-linux-gnu.so
           plugins.file_tracers: -none-
            plugins.configurers: -none-
      plugins.context_switchers: -none-
              configs_attempted: /tmp/pytest-of-root/pytest-0/popen-gw7/t23/.coveragerc
                                 /tmp/pytest-of-root/pytest-0/popen-gw7/t23/setup.cfg
                                 /tmp/pytest-of-root/pytest-0/popen-gw7/t23/tox.ini
                                 /tmp/pytest-of-root/pytest-0/popen-gw7/t23/pyproject.toml
                   configs_read: -none-
                    config_file: None
                config_contents: -none-
                      data_file: /tmp/pytest-of-root/pytest-0/popen-gw7/t23/.coverage
                         python: 3.9.23 (main, Jul 25 2025, 00:00:00) [GCC 15.1.1 20250521 (Red Hat 15.1.1-2)]
                       platform: Linux-6.12.41-gentoo-dist-x86_64-with-glibc2.42
                 implementation: CPython
                          build: main
                                 Jul 25 2025 00:00:00
                    gil_enabled: True
                     executable: /coveragepy/.tox/py39/bin/python
                   def_encoding: utf-8
                    fs_encoding: utf-8
                            pid: 1641
                            cwd: /tmp/pytest-of-root/pytest-0/popen-gw7/t23
                           path: /tmp/pytest-of-root/pytest-0/popen-gw7/t23
                                 /usr/lib64/python39.zip
                                 /usr/lib64/python3.9
                                 /usr/lib64/python3.9/lib-dynload
                                 /coveragepy/.tox/py39/lib64/python3.9/site-packages
                                 /coveragepy/.tox/py39/lib/python3.9/site-packages
                    environment: COVERAGE_RUN = true
                                 COVERAGE_TESTING = True
                                 HOME = /root
                                 PYTEST_CURRENT_TEST = tests/test_process.py::CoverageCoreTest::test_core_default (call)
                                 PYTEST_VERSION = 8.4.2
                                 PYTEST_XDIST_TESTRUNUID = 096ea5d3b85e456996338b8b35ef51d4
                                 PYTEST_XDIST_WORKER = gw7
                                 PYTEST_XDIST_WORKER_COUNT = 16
                                 PYTHONHASHSEED = 856143455
                                 PYTHONIOENCODING = UTF-8
                                 PYTHONWARNDEFAULTENCODING = 
                                 PYTHON_COLORS = 0
                   command_line: /coveragepy/.tox/py39/bin/coverage run --debug=sys numbers.py
                 coverage_paths: /coveragepy/.tox/py39/lib64/python3.9/site-packages/coverage/tests
                                 /coveragepy/.tox/py39/lib64/python3.9/site-packages/coverage
                   stdlib_paths: /usr/lib64/python3.9
              third_party_paths: /coveragepy/.tox/py39/bin
                                 /coveragepy/.tox/py39/lib64/python3.9/site-packages
                                 /root/.local/bin
                                 /coveragepy/.tox/py39/lib/python3.9/site-packages
                                 /root/.local/lib64/python3.9/site-packages
                                 /root/.local/lib/python3.9/site-packages
                                 /coveragepy/.tox/py39/lib/python
                   source_match: -none-
              source_pkgs_match: -none-
                  include_match: -none-
                     omit_match: -none-
                    cover_match: /coveragepy/.tox/py39/lib64/python3.9/site-packages/coverage
                                 /coveragepy/.tox/py39/lib64/python3.9/site-packages/coverage/tests
                    pylib_match: /usr/lib64/python3.9
                    third_match: /coveragepy/.tox/py39/bin
                                 /coveragepy/.tox/py39/lib64/python3.9/site-packages
                                 /coveragepy/.tox/py39/lib/python
                                 /coveragepy/.tox/py39/lib/python3.9/site-packages
                                 /root/.local/bin
                                 /root/.local/lib64/python3.9/site-packages
                                 /root/.local/lib/python3.9/site-packages
          source_in_third_match: -none-
         sqlite3_sqlite_version: 3.50.2
             sqlite3_temp_store: 0
        sqlite3_compile_options: ATOMIC_INTRINSICS=1, COMPILER=gcc-15.1.1 20250521 (Red Hat 15.1.1-2),
                                 DEFAULT_AUTOVACUUM, DEFAULT_CACHE_SIZE=-2000, DEFAULT_FILE_FORMAT=4,
                                 DEFAULT_JOURNAL_SIZE_LIMIT=-1, DEFAULT_MMAP_SIZE=0, DEFAULT_PAGE_SIZE=4096,
                                 DEFAULT_PCACHE_INITSZ=20, DEFAULT_RECURSIVE_TRIGGERS,
                                 DEFAULT_SECTOR_SIZE=4096, DEFAULT_SYNCHRONOUS=2,
                                 DEFAULT_WAL_AUTOCHECKPOINT=1000, DEFAULT_WAL_SYNCHRONOUS=2,
                                 DEFAULT_WORKER_THREADS=0, DIRECT_OVERFLOW_READ, DISABLE_DIRSYNC,
                                 ENABLE_COLUMN_METADATA, ENABLE_DBPAGE_VTAB, ENABLE_DBSTAT_VTAB,
                                 ENABLE_FTS3, ENABLE_FTS3_PARENTHESIS, ENABLE_FTS4, ENABLE_FTS5,
                                 ENABLE_MATH_FUNCTIONS, ENABLE_PREUPDATE_HOOK, ENABLE_RTREE, ENABLE_SESSION,
                                 ENABLE_UNLOCK_NOTIFY, HAVE_ISNAN, MALLOC_SOFT_LIMIT=1024, MAX_ATTACHED=10,
                                 MAX_COLUMN=2000, MAX_COMPOUND_SELECT=500, MAX_DEFAULT_PAGE_SIZE=8192,
                                 MAX_EXPR_DEPTH=1000, MAX_FUNCTION_ARG=1000, MAX_LENGTH=1000000000,
                                 MAX_LIKE_PATTERN_LENGTH=50000, MAX_MMAP_SIZE=0x7fff0000,
                                 MAX_PAGE_COUNT=0xfffffffe, MAX_PAGE_SIZE=65536, MAX_SQL_LENGTH=1000000000,
                                 MAX_TRIGGER_DEPTH=1000, MAX_VARIABLE_NUMBER=32766, MAX_VDBE_OP=250000000,
                                 MAX_WORKER_THREADS=8, MUTEX_PTHREADS, SECURE_DELETE, SYSTEM_MALLOC,
                                 TEMP_STORE=1, THREADSAFE=1
-- end -------------------------------------------------------
123 456

__________________________________________________ CoverageCoreTest.test_core_request_ctrace_but_missing ___________________________________________________
[gw8] linux -- Python 3.9.23 /coveragepy/.tox/py39/bin/python

self = <tests.test_process.CoverageCoreTest object at 0x7fce58d28070>

    @pytest.mark.skipif(has_ctracer, reason="CTracer needs to be missing")
    def test_core_request_ctrace_but_missing(self) -> None:
        self.del_environ("COVERAGE_CORE")
        self.make_file(".coveragerc", "[run]\ncore = ctrace\n")
        self.make_file("numbers.py", "print(123, 456)")
        out = self.run_command("coverage run --debug=sys,pybehave numbers.py")
        assert out.endswith("123 456\n")
        core = re_line(r" core:", out).strip()
>       assert core == "core: PyTracer"
E       AssertionError: assert 'core: CTracer' == 'core: PyTracer'
E         
E         - core: PyTracer
E         ?       ^^
E         + core: CTracer
E         ?       ^

/coveragepy/tests/test_process.py:1363: AssertionError
------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------
-- sys -------------------------------------------------------
               coverage_version: 7.10.7a0.dev1
                coverage_module: /coveragepy/.tox/py39/lib64/python3.9/site-packages/coverage/__init__.py
                           core: CTracer
                        CTracer: available from /coveragepy/.tox/py39/lib64/python3.9/site-packages/coverage/tracer.cpython-39-x86_64-linux-gnu.so
           plugins.file_tracers: -none-
            plugins.configurers: -none-
      plugins.context_switchers: -none-
              configs_attempted: /tmp/pytest-of-root/pytest-0/popen-gw8/t29/.coveragerc
                   configs_read: /tmp/pytest-of-root/pytest-0/popen-gw8/t29/.coveragerc
                    config_file: /tmp/pytest-of-root/pytest-0/popen-gw8/t29/.coveragerc
                config_contents: b'[run]\ncore = ctrace\n'
                      data_file: /tmp/pytest-of-root/pytest-0/popen-gw8/t29/.coverage
                         python: 3.9.23 (main, Jul 25 2025, 00:00:00) [GCC 15.1.1 20250521 (Red Hat 15.1.1-2)]
                       platform: Linux-6.12.41-gentoo-dist-x86_64-with-glibc2.42
                 implementation: CPython
                          build: main
                                 Jul 25 2025 00:00:00
                    gil_enabled: True
                     executable: /coveragepy/.tox/py39/bin/python
                   def_encoding: utf-8
                    fs_encoding: utf-8
                            pid: 1668
                            cwd: /tmp/pytest-of-root/pytest-0/popen-gw8/t29
                           path: /tmp/pytest-of-root/pytest-0/popen-gw8/t29
                                 /usr/lib64/python39.zip
                                 /usr/lib64/python3.9
                                 /usr/lib64/python3.9/lib-dynload
                                 /coveragepy/.tox/py39/lib64/python3.9/site-packages
                                 /coveragepy/.tox/py39/lib/python3.9/site-packages
                    environment: COVERAGE_RUN = true
                                 COVERAGE_TESTING = True
                                 HOME = /root
                                 PYTEST_CURRENT_TEST = tests/test_process.py::CoverageCoreTest::test_core_request_ctrace_but_missing (call)
                                 PYTEST_VERSION = 8.4.2
                                 PYTEST_XDIST_TESTRUNUID = 096ea5d3b85e456996338b8b35ef51d4
                                 PYTEST_XDIST_WORKER = gw8
                                 PYTEST_XDIST_WORKER_COUNT = 16
                                 PYTHONHASHSEED = 856143455
                                 PYTHONIOENCODING = UTF-8
                                 PYTHONWARNDEFAULTENCODING = 
                                 PYTHON_COLORS = 0
                   command_line: /coveragepy/.tox/py39/bin/coverage run --debug=sys,pybehave numbers.py
                 coverage_paths: /coveragepy/.tox/py39/lib64/python3.9/site-packages/coverage/tests
                                 /coveragepy/.tox/py39/lib64/python3.9/site-packages/coverage
                   stdlib_paths: /usr/lib64/python3.9
              third_party_paths: /coveragepy/.tox/py39/bin
                                 /coveragepy/.tox/py39/lib64/python3.9/site-packages
                                 /root/.local/bin
                                 /coveragepy/.tox/py39/lib/python3.9/site-packages
                                 /root/.local/lib64/python3.9/site-packages
                                 /root/.local/lib/python3.9/site-packages
                                 /coveragepy/.tox/py39/lib/python
                   source_match: -none-
              source_pkgs_match: -none-
                  include_match: -none-
                     omit_match: -none-
                    cover_match: /coveragepy/.tox/py39/lib64/python3.9/site-packages/coverage
                                 /coveragepy/.tox/py39/lib64/python3.9/site-packages/coverage/tests
                    pylib_match: /usr/lib64/python3.9
                    third_match: /coveragepy/.tox/py39/bin
                                 /coveragepy/.tox/py39/lib64/python3.9/site-packages
                                 /coveragepy/.tox/py39/lib/python
                                 /coveragepy/.tox/py39/lib/python3.9/site-packages
                                 /root/.local/bin
                                 /root/.local/lib64/python3.9/site-packages
                                 /root/.local/lib/python3.9/site-packages
          source_in_third_match: -none-
         sqlite3_sqlite_version: 3.50.2
             sqlite3_temp_store: 0
        sqlite3_compile_options: ATOMIC_INTRINSICS=1, COMPILER=gcc-15.1.1 20250521 (Red Hat 15.1.1-2),
                                 DEFAULT_AUTOVACUUM, DEFAULT_CACHE_SIZE=-2000, DEFAULT_FILE_FORMAT=4,
                                 DEFAULT_JOURNAL_SIZE_LIMIT=-1, DEFAULT_MMAP_SIZE=0, DEFAULT_PAGE_SIZE=4096,
                                 DEFAULT_PCACHE_INITSZ=20, DEFAULT_RECURSIVE_TRIGGERS,
                                 DEFAULT_SECTOR_SIZE=4096, DEFAULT_SYNCHRONOUS=2,
                                 DEFAULT_WAL_AUTOCHECKPOINT=1000, DEFAULT_WAL_SYNCHRONOUS=2,
                                 DEFAULT_WORKER_THREADS=0, DIRECT_OVERFLOW_READ, DISABLE_DIRSYNC,
                                 ENABLE_COLUMN_METADATA, ENABLE_DBPAGE_VTAB, ENABLE_DBSTAT_VTAB,
                                 ENABLE_FTS3, ENABLE_FTS3_PARENTHESIS, ENABLE_FTS4, ENABLE_FTS5,
                                 ENABLE_MATH_FUNCTIONS, ENABLE_PREUPDATE_HOOK, ENABLE_RTREE, ENABLE_SESSION,
                                 ENABLE_UNLOCK_NOTIFY, HAVE_ISNAN, MALLOC_SOFT_LIMIT=1024, MAX_ATTACHED=10,
                                 MAX_COLUMN=2000, MAX_COMPOUND_SELECT=500, MAX_DEFAULT_PAGE_SIZE=8192,
                                 MAX_EXPR_DEPTH=1000, MAX_FUNCTION_ARG=1000, MAX_LENGTH=1000000000,
                                 MAX_LIKE_PATTERN_LENGTH=50000, MAX_MMAP_SIZE=0x7fff0000,
                                 MAX_PAGE_COUNT=0xfffffffe, MAX_PAGE_SIZE=65536, MAX_SQL_LENGTH=1000000000,
                                 MAX_TRIGGER_DEPTH=1000, MAX_VARIABLE_NUMBER=32766, MAX_VDBE_OP=250000000,
                                 MAX_WORKER_THREADS=8, MUTEX_PTHREADS, SECURE_DELETE, SYSTEM_MALLOC,
                                 TEMP_STORE=1, THREADSAFE=1
-- pybehave --------------------------------------------------
                        CPYTHON: True
                            GIL: True
                          LINUX: True
                          MACOS: False
                        METACOV: False
                           PYPY: False
                    PYPYVERSION: (0,)
                      PYVERSION: (3, 9, 23, 'final', 0, 0)
                SHIPPING_WHEELS: True
                 SYSMON_DEFAULT: False
                        TESTING: True
                        WINDOWS: False
              branch_right_left: False
           deferred_annotations: False
        docstring_only_function: True
              exit_through_with: False
       exit_with_through_ctxmgr: False
                 fstring_syntax: False
             keep_constant_test: False
                 lasti_is_yield: True
                     match_case: False
             module_firstline_1: False
                omit_after_jump: False
              omit_after_return: False
              optimize_if_debug: True
          optimize_if_not_debug: 2
  optimize_unreachable_try_else: False
                         pep626: False
                         pep669: False
                  soft_keywords: False
-- end -------------------------------------------------------
123 456

__________________________________________________________ VirtualenvTest.test_bug_888[coverage] ___________________________________________________________
[gw1] linux -- Python 3.9.23 /coveragepy/.tox/py39/bin/python

self = <tests.test_venv.VirtualenvTest object at 0x7f6d6d497640>, coverage_command = 'coverage'

    def test_bug_888(self, coverage_command: str) -> None:
        out = run_in_venv(
            coverage_command + " run --source=bug888/app,bug888/plugin bug888/app/testcov/main.py",
        )
        # When the test fails, the output includes "Already imported a file that will be measured"
>       assert out == "Plugin here\n"
E       AssertionError: assert 'Plugin here\nPlugin here\n' == 'Plugin here\n'
E         
E           Plugin here
E         + Plugin here

/coveragepy/tests/test_venv.py:405: AssertionError
------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------
Plugin here
Plugin here

_____________________________________________________ VirtualenvTest.test_bug_888[python -m coverage] ______________________________________________________
[gw1] linux -- Python 3.9.23 /coveragepy/.tox/py39/bin/python

self = <tests.test_venv.VirtualenvTest object at 0x7f6d6d497700>, coverage_command = 'python -m coverage'

    def test_bug_888(self, coverage_command: str) -> None:
        out = run_in_venv(
            coverage_command + " run --source=bug888/app,bug888/plugin bug888/app/testcov/main.py",
        )
        # When the test fails, the output includes "Already imported a file that will be measured"
>       assert out == "Plugin here\n"
E       AssertionError: assert 'Plugin here\nPlugin here\n' == 'Plugin here\n'
E         
E           Plugin here
E         + Plugin here

/coveragepy/tests/test_venv.py:405: AssertionError
------------------------------------------------------------------- Captured stdout call -------------------------------------------------------------------
Plugin here
Plugin here

================================================================= short test summary info ==================================================================
FAILED tests/test_process.py::CoverageCoreTest::test_core_default - AssertionError: assert 'core: CTracer' == 'core: PyTracer'
FAILED tests/test_process.py::CoverageCoreTest::test_core_request_ctrace_but_missing - AssertionError: assert 'core: CTracer' == 'core: PyTracer'
FAILED tests/test_venv.py::VirtualenvTest::test_bug_888[coverage]@virtualenv_test - AssertionError: assert 'Plugin here\nPlugin here\n' == 'Plugin here\n'
FAILED tests/test_venv.py::VirtualenvTest::test_bug_888[python -m coverage]@virtualenv_test - AssertionError: assert 'Plugin here\nPlugin here\n' == 'Plugin here\n'
4 failed, 1442 passed, 62 skipped, 1 xfailed in 16.49s
py39: exit 1 (16.88 seconds) /coveragepy> python igor.py test_with_core pytrace pid=155
py39: FAIL ✖ in 29.29 seconds

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions