Skip to content

Commit 5fe0bc5

Browse files
rwgkrwgk
committed
Add leak_detach path for internals after shutdown
Py_IsInitialized/Py_IsFinalizing are runtime-wide, not per-interpreter, and Py_IsFinalizing stays set after Py_Finalize until the next init. Using those checks to decide whether to DECREF internals can skip cleanup or touch the C-API after a specific interpreter is gone. Add internals::leak_detach() and local_internals::leak_detach() to clear the owned PyType/PyObject pointers without calling into Python, and invoke that from internals_pp_manager::destroy() before deleting the pp in the post- Py_Finalize/Py_EndInterpreter paths. The destructors now always Py_CLEAR when invoked during state-dict teardown, preserving cleanup of the pybind11 heap types while avoiding UB in late-destroy scenarios. Cursor GPT-5.2 Codex Extra High ┌──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ → Could you please read this file for context: /wrk/logs/gh_pr_info_pybind_pybind11_5958_2026-01-19+100747.md │ │ │ │ In my last comment there I wrote: I'll point Cursor to this, the pybind11 sources, and the CPython 3.14 sources, and ask it to look very thoroughly to figure out what is the │ │ best achievable solution in terms of avoiding UB but leaking as little as possible. │ │ │ │ The up-to-date pybind11 sources are right here (master branch). │ │ │ │ The CPython sources are here: │ │ │ │ smc120-0009.ipp2a2.colossus.nvidia.com:/wrk/clone/cpython $ git branch │ │ * 3.14 │ │ main │ │ v3.14.2_release │ │ │ │ JIC we need this for your analysis: │ │ │ │ I last git pulled the CPython 3.14 branch yesterday and built from sources using this script: │ │ │ │ /home/rgrossekunst/rwgk_config/bin/build_cpython_from_git_twice.sh │ │ │ │ The installations are here: │ │ │ │ /wrk/cpython_installs/3.14_branch_23e3c0ae867_default │ │ /wrk/cpython_installs/3.14_branch_23e3c0ae867_freethreaded │ │ │ │ Could you please "look very thoroughly to figure out what is the best achievable solution in terms of avoiding UB but leaking as little as possible.", also taking Joshua's "I │ │ think the correct solution is a internals::leak_detach() method" comment into account? │ │ │ │ Please use the cursor_workspace/ subdirectly here for all intermediate/temporary files. Do NOT use /tmp. │ │ │ │ If you want to rebuild the pybind11 unit tests for your analysis, please do NOT use cmake. Use this command instead: │ │ │ │ cd /wrk/bld/pybind11_gcc_v3.14.2_df793163d58_default/ && scons -j $(nproc) && TestVenv/bin/python3 ../../clone/pybind11_scons/run_tests.py ../../forked/pybind11 24 │ │ │ │ Currently I want to focus on the "default" cpython build. We can leave any "freethreaded" considerations for later. │ └──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
1 parent da6e071 commit 5fe0bc5

File tree

1 file changed

+29
-25
lines changed

1 file changed

+29
-25
lines changed

‎include/pybind11/detail/internals.h‎

Lines changed: 29 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -195,14 +195,6 @@ struct override_hash {
195195

196196
using instance_map = std::unordered_multimap<const void *, instance *>;
197197

198-
inline bool is_interpreter_alive() {
199-
#if PY_VERSION_HEX < 0x030D0000
200-
return Py_IsInitialized() != 0 || _Py_IsFinalizing() != 0;
201-
#else
202-
return Py_IsInitialized() != 0 || Py_IsFinalizing() != 0;
203-
#endif
204-
}
205-
206198
#ifdef Py_GIL_DISABLED
207199
// Wrapper around PyMutex to provide BasicLockable semantics
208200
class pymutex {
@@ -316,16 +308,21 @@ struct internals {
316308
internals(internals &&other) = delete;
317309
internals &operator=(const internals &other) = delete;
318310
internals &operator=(internals &&other) = delete;
311+
void leak_detach() noexcept {
312+
// Used when internals must be destroyed after interpreter teardown.
313+
// Avoid touching the Python C-API by clearing pointers only.
314+
instance_base = nullptr;
315+
default_metaclass = nullptr;
316+
static_property_type = nullptr;
317+
}
318+
319319
~internals() {
320-
// Normally this destructor runs during interpreter finalization and it may DECREF things.
321-
// In odd finalization scenarios it might end up running after the interpreter has
322-
// completely shut down, In that case, we should not decref these objects because pymalloc
323-
// is gone.
324-
if (is_interpreter_alive()) {
325-
Py_CLEAR(instance_base);
326-
Py_CLEAR(default_metaclass);
327-
Py_CLEAR(static_property_type);
328-
}
320+
// Destruction is expected to run while the interpreter is still intact
321+
// (e.g., during state-dict teardown). If we must destroy after shutdown,
322+
// leak_detach() must have been called first.
323+
Py_CLEAR(instance_base);
324+
Py_CLEAR(default_metaclass);
325+
Py_CLEAR(static_property_type);
329326
}
330327
};
331328

@@ -344,14 +341,16 @@ struct local_internals {
344341
std::forward_list<ExceptionTranslator> registered_exception_translators;
345342
PyTypeObject *function_record_py_type = nullptr;
346343

344+
void leak_detach() noexcept {
345+
// Used when local internals must be destroyed after interpreter teardown.
346+
function_record_py_type = nullptr;
347+
}
348+
347349
~local_internals() {
348-
// Normally this destructor runs during interpreter finalization and it may DECREF things.
349-
// In odd finalization scenarios it might end up running after the interpreter has
350-
// completely shut down, In that case, we should not decref these objects because pymalloc
351-
// is gone.
352-
if (is_interpreter_alive()) {
353-
Py_CLEAR(function_record_py_type);
354-
}
350+
// Destruction is expected to run while the interpreter is still intact
351+
// (e.g., during state-dict teardown). If we must destroy after shutdown,
352+
// leak_detach() must have been called first.
353+
Py_CLEAR(function_record_py_type);
355354
}
356355
};
357356

@@ -716,13 +715,18 @@ class internals_pp_manager {
716715
// this could be called without an active interpreter, just use what was cached
717716
if (!tstate || tstate->interp == last_istate_tls()) {
718717
auto tpp = internals_p_tls();
719-
718+
if (tpp && tpp->get()) {
719+
tpp->get()->leak_detach();
720+
}
720721
delete tpp;
721722
}
722723
unref();
723724
return;
724725
}
725726
#endif
727+
if (internals_singleton_pp_ && internals_singleton_pp_->get()) {
728+
internals_singleton_pp_->get()->leak_detach();
729+
}
726730
delete internals_singleton_pp_;
727731
unref();
728732
}

0 commit comments

Comments
 (0)