Reflection on the current (as of 2025-07-03) governance and decision making

[mroeschke]

In light of recent challenges and miscommunications with PDEP 15 voting and the PR to add IO engine registration, I just wanted to reflect on how the our decision making process is serving the project, specifically the pandas-dev/pandas project. Might be a good topic to discuss the next time the governance team meets.

To summarize: I feel like we've introduced a lot of process for a project with a relatively low development engagement.

While PDEP-1 specifies how "major" decisions should be made, I think:

1. The process and rules for making/following a new PDEP are probably heavy-handed with the amount of active contributors

   • I think the PDEP 15 processes showed maybe we could use more leeway for "common sense" instead of going with the motions of the rules (e.g. "this PDEP doesn't seem relevant anymore, anyone object to just close this PDEP").
   • I feel there is apprehension from the core team of needing to go through the PDEP process, especially if one thinks a topic doesn't need a PDEP and another suggests it does therefore adding a non-trivial amount of additional effort. This is probably due to the ambiguity of 2.

2. Should probably address process for making non-major decisions i.e. merging PRs

   • I think the IO registration issue showed there's still some ad-hoc, yet varied conventions we follow when a new PDEP is not needed for merging a PR.
   • The convention in https://pandas.pydata.org/docs/development/maintaining.html#merging-pull-requests should probably be amended to PDEP 1, but also clarifying if/how an uncompromising -1 can be overruled (e.g. with a non-PDEP vote of sorts)

[rhshadrach]

For the first bullet point of 1, I see nothing in our governance that did not allow us to just close the PDEP without a vote. If any core member desires for and calls for a vote following the proper procedure, then I think we should oblige. That is what was done in this instance.

[WillAyd]

I think as a team if we tried to evaluate the PDEP process it would skew pretty negative. From my perspective, the only PDEP I would label as a "success" (i.e. the team collaborated to design something) was PDEP-14, although I am not sure I can fully express how that one worked while others did not.

Recently we've had anywhere from 0-2 people actively contributing to the project at a time, and that has trended down sharply over the past few years. I think the PDEP process Marc tried to go through and his recent PR have to be extremely discouraging, because it opens a contributor up to feedback that is almost assuredly negative and rarely helpful. The way PRs make their way into the code base now are by flying under the radar and avoiding as much review from other team members, which I think is unfortunate.

[jbrockmendel]

FWIW on the last dev call it was me, Joris, and Irv, and we kind of had to throw up our hands on discussing Big Topics because we wouldn't really be able to decide anything on the call. In the old days, the calls were where we decided on things that didn't have consensus on GH.

[jorisvandenbossche]

(I am travelling at the moment, and will probably not (much) respond the rest of the week, also won't be able to join the dev meeting later today)

My personal experience with the PDEPs that I have been involved with (CoW, string dtype, inplace, ..) has been generally positive, although they were all about bigger topics (compared to adding a function or plugin). For those bigger topics I feel they certainly have provided a clearer process. It takes a lot of time for sure, but such bigger topics take a lot of time anyway. And spending that time with a clear process is better than without .. (eg when I spent time on array manager and nullable dtypes before the PDEP process).

[WillAyd]

Thanks @jorisvandenbossche for some counterpoints. Regarding the CoW PDEP, was that not just a formalization of work that had already been done?

Not challenging the point of view that it was successful, but I am just trying to figure out what makes some PDEPs work well and others not. Part of me wonders if there is preliminary work that should be done on a PDEP that makes it easier, versus coming in carte blanche and attempting to set a roadmap.