Some miscellaneous fixups

xgupta · xgupta · commit 3011a8961ca6 · 2026-05-20T21:01:01.000+05:30
diff --git a/llvm/utils/patch-coverage/README.md b/llvm/utils/patch-coverage/README.md
@@ -1,10 +1,8 @@
 # Patch-Based Code Coverage (LLVM Git Integration)
 
 This tool provides patch-level code coverage for LLVM development workflows.
-It helps developers and reviewers understand which lines introduced by a patch are covered by tests and which are not.
-
-It integrates with LLVM’s infrastructure (`llvm-lit`, coverage instrumentation, and CI) and works both locally and in CI environments.
-
+It helps developers and reviewers understand which lines introduced by a patch
+are covered by tests and which are not.
 
 ## Overview
 
@@ -21,23 +19,14 @@ Given recent commits, the tool:
    * Covered lines introduced by the patch
    * Uncovered lines introduced by the patch
 
-
-## Key Features
-
-* Patch-aware coverage (only modified lines)
-* Selective test execution (only affected tests)
-* Supports:
-
-  * `llvm-lit` tests
-  * LLVM unit tests
-* Uses allowlist-based instrumentation for faster builds
-* Avoids unnecessary rebuilds via patch hashing
-
-
 ## Setup
 
 ### 1. Build LLVM (standard workflow)
 
+If you have an existing LLVM build, point to it with `-b`. If not, skip this step —
+the tool will automatically configure and build LLVM in the specified build directory
+with a minimal configuration when first run.
+
 ```bash
 git clone https://github.com/llvm/llvm-project.git
 cd llvm-project
@@ -104,21 +93,69 @@ git patch-coverage -b build -n 3 opt llvm/test
 | ------------------------------ | ----------------------------------------------------------- |
 | `-n, --num-commits`            | Number of recent commits to analyze (default: 1)            |
 | `--projects`                   | LLVM projects to enable (e.g. clang;mlir (default: None)    |
-| `-i, --instrumented-build-dir` | Use pre-built instrumented directory (Deafult: `build_inst`)|               
+| `-i, --instrumented-build-dir` | Use pre-built instrumented directory (Dafault: `build_inst`)| 
 | `-b, --build-dir`              | Path to LLVM build directory (default: `build`)             |
 | `<test-path>`                  | One or more test suite paths (default: `build/test`)        |
 
 
 
 ## How It Works
-
-
+### 1. Patch Extraction
+The tool runs `git diff HEAD~N HEAD` to generate a patch from the last N commits.
+It parses the diff to extract:
+- **Source files** — modified `.cpp`, `.c`, or `.mm` files outside test directories
+- **Test files** — modified lit tests and unit tests within the patch
+- **Modified lines** — the exact line numbers added or changed in each source file
+
+### 2. Allowlist Generation
+To avoid instrumenting the entire codebase, the tool generates an allowlist
+containing only the source files present in the patch:   
+source:/path/to/modified/file.cpp=allow  
+default:skip 
+
+This is passed to the compiler via `-fprofile-list`, so profiling overhead is
+scoped only to what changed.
+
+### 3. Instrumented Build
+The tool configures a second build directory (`build_inst`) with:
+- `LLVM_BUILD_INSTRUMENTED_COVERAGE=ON` — enables source-based coverage
+- `LLVM_INDIVIDUAL_TEST_COVERAGE=ON` — generates per-test `.profraw` files
+- The allowlist flags from step 2
+
+Rebuilds are skipped if the patch hash (SHA-256 of the diff) matches the hash
+from the last successful build, stored in `build_inst/.last_patch_hash`.
+
+### 4. Test Execution
+Only tests modified in the patch are run, against the instrumented binary.
+Each test writes its coverage data to a dedicated `.profraw` file under
+`build_inst/profiles/`, named by a hash of the test path to avoid collisions:
+
+- **Lit tests** — run via `llvm-lit` with `LLVM_PROFILE_FILE` set per test
+- **Unit tests** — run directly as binaries with `LLVM_PROFILE_FILE` set
+
+### 5. Coverage Processing
+For each `.profraw` file collected during test execution:
+1. `llvm-profdata merge` converts it to a `.profdata` file
+2. `llvm-cov show` produces a line-level coverage report for each patched
+   source file, using the instrumented binary as the symbol source
+
+### 6. Reporting
+The tool cross-references coverage results against the modified lines extracted
+in step 1. A line is considered:
+- **Covered** — if it was executed by at least one test
+- **Uncovered** — if it appears in the patch but was never executed
+- **Effectively covered** — if it was uncovered in some tests but covered in
+  others (union of all runs wins)
+
+Results are printed to the terminal in diff style with color coding, and the
+full details are written to `build_inst/patch_coverage.log`.
 
 ## Output
 
 ### `<build_inst>/patch_coverage.log`
 
 ```
+...
 Modified File: llvm/lib/IR/Example.cpp
   Covered Line: 42 : foo();
   Uncovered Line: 45 : bar();
@@ -127,20 +164,22 @@ Modified File: llvm/lib/IR/Example.cpp
 ### Summary on Standard Output
 
 ```
-[code-coverage] Common uncovered lines for llvm/lib/IR/Example.cpp:
-  Line 45: bar();
+[code-coverage] Coverage report for llvm/lib/IR/Example.cpp:
+  Line 42  :   foo();
+  Line 45  :   bar();
+
 ```
 
 ## Troubleshooting
 
 ### No coverage generated
 
-* Ensure instrumented build succeeded and tests run
-* Check `.profraw` files in `build_inst` directory
+* Ensure instrumented build succeeded and tests run.
+* Check `.profraw` files in `build_inst` directory.
 
 ### Rebuild skipped unexpectedly
 
 * Delete `binary` to force rebuild
 
 ### For any other doubt
-* Check `<build_inst>/patch_coverage.log`. It has full logging of tool
+* Check `<build_inst>/patch_coverage.log`. It has full logging of tool.
diff --git a/llvm/utils/patch-coverage/main.py b/llvm/utils/patch-coverage/main.py
@@ -157,7 +157,7 @@ def main():
         mark_build_success(inst_build_dir, patch_path)
     else:
         print("\n[patch-coverage] Skipping patch coverage (no changes)")
-        sys.exit(1)
+        sys.exit(0)
 
     # Run all the test cases of patch with instrumented binary.
     inst_lit_path = os.path.join(inst_build_dir, "bin/llvm-lit")
diff --git a/llvm/utils/patch-coverage/process.py b/llvm/utils/patch-coverage/process.py
@@ -1,86 +1,97 @@
 import subprocess
 import sys
 import os
+from concurrent.futures import ThreadPoolExecutor, as_completed
 
 from utils import classify_tests
 from utils import log
-from utils import target_name
 
-# TODO: Can we process parallelly
 
-
-def process_coverage_data(
-    source_files, inst_build_dir, lit_binary, unit_binary, patch_path
+def process_single_profraw(
+    profraw_file, source_files, binary, llvm_profdata, llvm_cov
 ):
-    coverage_files = {}
+    results = {}
 
-    try:
-        os.chdir(inst_build_dir)
-
-        unit_tests, lit_tests = classify_tests(patch_path)
-        if not (unit_tests or lit_tests):
-            return coverage_files
+    profdata_output = os.path.splitext(profraw_file)[0] + ".profdata"
 
-        binary = unit_binary if unit_tests else lit_binary
+    log("\nProfraw File:", profraw_file)
+    log("Profdata File:", profdata_output)
 
-        for root, dirs, files in os.walk("."):
-            for file in files:
-                if os.path.basename(file) == "default.profraw":
-                    continue
+    subprocess.check_call(
+        [llvm_profdata, "merge", "-o", profdata_output, profraw_file]
+    )
+    log(f"Converted {profraw_file} to {profdata_output}")
 
-                if file.endswith(".profraw"):
-                    profraw_file = os.path.abspath(os.path.join(root, file))
-                    profdata_output = os.path.abspath(
-                        os.path.splitext(profraw_file)[0] + ".profdata"
-                    )
+    for cpp_file in source_files:
+        output_file = (
+            os.path.splitext(profdata_output)[0]
+            + f"_{os.path.basename(cpp_file)}.txt"
+        )
 
-                    log("\nProfraw File:", profraw_file)
-                    log("Profdata File:", profdata_output)
+        llvm_cov_cmd = [
+            llvm_cov, "show",
+            "-instr-profile", profdata_output,
+            binary,
+            "--format=text",
+            cpp_file,
+        ]
 
-                    llvm_profdata_cmd = [
-                        "./bin/llvm-profdata",
-                        "merge",
-                        "-o",
-                        profdata_output,
-                        profraw_file,
-                    ]
+        with open(output_file, "w") as output:
+            subprocess.check_call(llvm_cov_cmd, stdout=output)
 
-                    subprocess.check_call(llvm_profdata_cmd)
+        log(f"Processed file saved as: {output_file}")
+        results.setdefault(cpp_file, []).append(output_file)
 
-                    log(f"Converted {profraw_file} to {profdata_output}")
+    return results
 
-                    for cpp_file in source_files:
-                        output_file = (
-                            os.path.splitext(profdata_output)[0]
-                            + f"_{cpp_file.replace('/', '_')}.txt"
-                        )
 
-                        parent_directory = os.path.dirname(os.getcwd())
-                        abs_cpp_file = os.path.abspath(
-                            os.path.join(parent_directory, cpp_file)
-                        )
+def process_coverage_data(
+    source_files, inst_build_dir, lit_binary, unit_binary, patch_path
+):
+    coverage_files = {}
 
-                        llvm_cov_cmd = [
-                            "./bin/llvm-cov",
-                            "show",
-                            "-instr-profile",
-                            profdata_output,
-                            binary,
-                            "--format=text",
-                            abs_cpp_file,
-                        ]
+    unit_tests, lit_tests = classify_tests(patch_path)
+    if not (unit_tests or lit_tests):
+        return coverage_files
 
-                        with open(output_file, "w") as output:
-                            subprocess.check_call(llvm_cov_cmd, stdout=output)
+    binary = unit_binary if unit_tests else lit_binary
+    llvm_profdata = os.path.join(inst_build_dir, "bin", "llvm-profdata")
+    llvm_cov = os.path.join(inst_build_dir, "bin", "llvm-cov")
+    profiles_dir = os.path.join(inst_build_dir, "profiles")
 
-                        log(f"Processed file saved as: {output_file}")
+    profraw_files = [
+        os.path.abspath(os.path.join(profiles_dir, f))
+        for f in os.listdir(profiles_dir)
+        if f.endswith(".profraw")
+    ]
 
-                        coverage_files.setdefault(abs_cpp_file, []).append(output_file)
+    try:
+        with ThreadPoolExecutor() as executor:
+            futures = {
+                executor.submit(
+                    process_single_profraw,
+                    profraw_file,
+                    source_files,
+                    binary,
+                    llvm_profdata,
+                    llvm_cov,
+                ): profraw_file
+                for profraw_file in profraw_files
+            }
+
+            for future in as_completed(futures):
+                profraw_file = futures[future]
+                try:
+                    result = future.result()
+                    for cpp_file, files in result.items():
+                        coverage_files.setdefault(cpp_file, []).extend(files)
+                except subprocess.CalledProcessError as e:
+                    log(f"Error processing {profraw_file}: {e}")
+                    sys.exit(1)
 
         log("\nConversion of profraw files to human-readable form is completed.\n")
-
         return coverage_files
 
-    except subprocess.CalledProcessError as e:
-        log("Error during profraw to profdata conversion:", e)
+    except Exception as e:
+        log("Error during coverage processing:", e)
         sys.exit(1)
diff --git a/llvm/utils/patch-coverage/test.py b/llvm/utils/patch-coverage/test.py
@@ -7,7 +7,7 @@
 from utils import log
 from utils import target_name
 
-# TODO: Can we run llvm-lit in batch mode
+# TODO: We should run llvm-lit in batch mode.
 
 
 def run_single_test_with_coverage(llvm_lit_path, test_path, inst_build_dir):
diff --git a/llvm/utils/patch-coverage/utils.py b/llvm/utils/patch-coverage/utils.py
@@ -41,15 +41,13 @@ def mark_build_success(build_dir, patch_path):
 
 
 def delete_profraw(inst_build_dir):
-    cwd = os.getcwd()
-    os.chdir(inst_build_dir)
-
-    command = 'find . -type f -name "*.profraw" -delete'
-    os.chdir(cwd)
-
+    abs_dir = os.path.abspath(inst_build_dir)
     try:
-        subprocess.run(command, shell=True, check=True)
-        log("Older '.profraw' files are successfully deleted.")
+        subprocess.run(
+            ["find", abs_dir, "-type", "f", "-name", "*.profraw", "-delete"],
+            check=True
+        )
+        log("Older '.profraw' files successfully deleted.")
     except subprocess.CalledProcessError as e:
         log(f"Error: {e}")
 
