gh-140727: Restructure profiling documentation for PEP 799 #142373
Conversation
PEP 799 introduces a new `profiling` package that reorganizes Python's profiling tools under a unified namespace. This commit adds the documentation structure to match: a main entry point (profiling.rst) that helps users choose between profilers, detailed docs for the tracing profiler (profiling-tracing.rst), and separated pstats documentation. The tracing profiler docs note that cProfile remains as a backward-compatible alias, so existing code continues to work. The pstats module gets its own page since it's used by both profiler types and deserves focused documentation.
The sampling profiler is new in Python 3.15 and works fundamentally differently from the tracing profiler. It observes programs from outside by periodically capturing stack snapshots, which means zero overhead on the profiled code. This makes it practical for production use where you can attach to live servers. The docs explain the key concepts (statistical vs deterministic profiling), provide quick examples upfront, document all output formats (pstats, flamegraph, gecko, heatmap), and cover the live TUI mode. The defaults table helps users understand what happens without any flags.
Add the new profiling module pages to the Debugging and Profiling toctree. The order places the main profiling.rst entry point first, followed by the two profiler implementations, then pstats, and finally the deprecated profile module last.
The pure Python profile module is deprecated in 3.15 and scheduled for removal in 3.17. Users should migrate to profiling.tracing (or use the cProfile alias which continues to work). The page now focuses on helping existing users migrate: it shows the old vs new import style, keeps the shared API reference since both modules have the same interface, and preserves the calibration docs for anyone still using the pure Python implementation during the transition period.
Point cProfile to profiling.tracing docs and add profiling.sampling to the list of modules with CLI interfaces. The old profile-cli label no longer exists after the documentation restructure.
Enable cross-references to the new profiling module documentation and update the CLI examples to use the current syntax with the attach subcommand. Also reference profiling.tracing instead of cProfile since that's the new canonical name.
|
Most of this PR is moving existing documentation into separate files rather than writing new content. The The actual new writing is in |
Co-authored-by: Stan Ulbrych <[email protected]>
Co-authored-by: Stan Ulbrych <[email protected]>
| :mod:`threading` module or implicitly through libraries that manage thread | ||
| pools. | ||
|
|
||
| By default, the profiler samples only the main thread. The ``--all-threads`` |
There was a problem hiding this comment.
Threading feels quite common. It feels like the option even existing implies there is some consequence of enabling it? State why someone might NOT want to sample all threads in the docs.
Perhaps out of scope for this docs PR, but do we actually want this default or should it be the opposite? Does the option then even need to exist?
There was a problem hiding this comment.
Threading feels quite common. It feels like the option even existing implies there is some consequence of enabling it? State why someone might NOT want to sample all threads in the docs.
Perhaps out of scope for this docs PR, but do we actually want this default or should it be the opposite? Does the option then even need to exist?
We can discuss this elsewhere, but the reason is that is more expensive. You are basically dividing the sampling rate by the number of threads. There are also some of important optimizations if you know that you only want to sample the main thread.
This also generates a lot more data so the result files are a bit bigger. We can chat about what we think the default should be but the option should exist so people can decide to dedicate all sampling budget to the main thread if they wish. This can make a big difference.
There was a problem hiding this comment.
State why someone might NOT want to sample all threads in the docs.
Good point 👍 Will do
|
Tachyon logo by @maigimenez ❤️ (she did the 3.11 and 3.10 cool release logos) |
Co-authored-by: Stan Ulbrych <[email protected]>
hugovk
left a comment
hugovk
left a comment
There was a problem hiding this comment.
I started adding suggestions via the GitHub UI but it was taking 4s to click the + and 6s to click "Add review comment" so I gave up and opened pablogsal#108 instead :/
|
Co-authored-by: Hugo van Kemenade <[email protected]>
6 participants
This PR restructures the profiling documentation to match the new
profilingpackage introduced by PEP 799.
The existing monolithic
profile.rstis split into focused pages:profiling.rst: Entry point that helps users choose between statistical
and deterministic profiling, with a comparison table and guidance on when to
use each approach.
profiling-tracing.rst: Documents the deterministic profiler (formerly
documented under cProfile). Notes that
cProfileremains as abackward-compatible alias.
profiling-sampling.rst: Documents the new statistical sampling
profiler. Includes quick examples upfront, explains the key concepts, covers
all output formats (pstats, flamegraph, gecko, heatmap), live mode, and
async-aware profiling. A defaults table shows what happens without any flags.
pstats.rst: Separated out since it's used by both profiler types.
profile.rst: Reduced to a deprecation stub with migration guidance for
users of the pure Python profiler.
The whatsnew entry is updated to link to the new documentation and reference
profiling.tracinginstead ofcProfile.📚 Documentation preview 📚: https://cpython-previews--142373.org.readthedocs.build/