Allow disabling function and class index pages in html reports

[pankajp]

Is your feature request related to a problem? Please describe.
In one of my projects, generating class and function index pages is taking more time than rest of the things including the full test code execution under coverage (timid) and html report generation upto index.html when it prints Wrote HTML report to html/index.html. After that there's a long wait until coverage process exits while it is generating class and index pages.

Describe the solution you'd like
Add a config option in [html] section to disable this class and function index page

Describe alternatives you've considered
Monkey-patching the code to no-op the coverage.html.HtmlReporter.write_region_index_pages

Additional context

[nedbat]

This sounds like a reasonable idea, but I'm surprised at your timings. How large is your project? How many files in the HTML report, how many classes, how many functions?

[pankajp]

The project has humongous python files with some of the larger ones being a couple of megabytes in size! The timings (3 min for test+index.html generation, +4 mins for class and function index generation) were for just a few tests and not the whole test suite.

[nedbat]

I don't suppose it's possible for me to experiment with your repo? It's hard to understand how the class/function pages could be slower than everything else.

[nedbat]

@devdanzin came up with a patch that seems to speed things up a lot.
@pankajp can you try this?

diff --git a/coverage/results.py b/coverage/results.py
index 6d28e73f..593267c2 100644
--- a/coverage/results.py
+++ b/coverage/results.py
@@ -123,9 +123,9 @@ class Analysis:
         the lines in `lines`.
         """

-        statements = {lno for lno in self.statements if lno in lines}
-        excluded = {lno for lno in self.excluded if lno in lines}
-        executed = {lno for lno in self.executed if lno in lines}
+        statements = self.statements & lines
+        excluded = self.excluded & lines
+        executed = self.executed & lines

         if self.has_arcs:
             arc_possibilities_set = {

[nedbat]

@pankajp also, are you using branch coverage, or just statement coverage?

[pankajp]

This patch helped with the performance, reducing html reporting time from 220s to 120s (also converting the no_branch = comprehension to &).
And you are right I'm using branch coverage so the most expensive line after this change is:

            arc_possibilities_set = {
                (a, b) for a, b in self.arc_possibilities_set
                if a in lines or b in lines
            }

I couldn't easily find an optimization for this line. If there is a way to filter the arcs only when reporting, then maybe we could get away with keeping the full arc_possibilities and only check the keys relevant to the narrowed analysis where needed.
From the profile report this one line is now >40% of total cost of html report! But the mentioned patch is already a huge improvement! Thanks @devdanzin and @nedbat

[nedbat]

I've refactored the narrowing code to avoid the performance problems in commit 0d5a112.

Technically, there are still some O(N**2) operations, but in the function and class reporting they are very fast. It should be a small fraction of the total time needed for HTML reporting. Let me know how it seems to you.

You can install it with:

python3 -m pip install git+https://github.com/nedbat/coveragepy@0d5a112fc54c1d5a07f3#egg=coverage==0.0

[nedbat]

This is now released as part of coverage 7.10.7.