major overhaul of pyjanitor

[ericmjl]

Leaving some notes, but @samukweku I'd love your thoughts here. With AI assistance, this overhaul I've been mulling finally feels a lot more approachable.

• 1. Switch from conda-based dev env to pixi-based dev env.
• 2. Automate patch releases on every merge, humans get to decide whether we do also a minor/major release or not based on whether the last changes merged were breaking or not.
• 3. Better dataframe-agnostic code, move to ibis/narwhals? but a lot of the stuff you've done with numba for performance may be in a tricky spot with this move.
• 4. Better AI harnesses so that easy features can be quickly implemented, still need review from humans.
• 5. Fully embrace syntactic sugar? We had a lot of deprecations, but I think those may be used by folks. Only risk is more code to maintain.
• 6. Parity with R janitor package.

[samukweku]

@ericmjl 👍

1. Yes let's do it
2. Yes let's do it
3. A number of our syntactic sugar is for pandas - e.g select_columns helps with easy selection in pandas - polars already has pl.col as well as the selectors module to cover this, which is why select_columns is not implemented for polars; the specific ones though are targeted? (e.g complete in pandas has a different approach to that in polars). Ibis is more SQL based; narwhals might be a better option. How narwhals handles that with some optimisations we do per library (pandas/polars) would probably require some discovery. There is also the numba part as well (not sure how many users rely on the numba implementation). There is also the work in janitor_rs for performance improvements - some discovery work would also be needed there to combine that with narwhals.
4. yes let's do it
5. part of the reasons we deprecated syntactic sugar was because it was already covered by pandas/polars so we dont have to maintain it - Deprecate functions ? #1045 . The less code to maintain the better.
6. Yes, let's do it

[ericmjl]

Okie dokes, the ones you've agreed to, I'm going to get the ball rolling!

[ericmjl]

As for points 3 and 5, let's stick with your points, I agree as well.

[samukweku]

i'm working on the rust angle ... if I can get that working, then I'll deprecate the numba support.