support the creation fo an actual plugin dependency tree and use its root for consideration/order

[RonnyPfannschmidt]

in pytest when we use conftests with pytest_plugins,
those specifications are not disabled when using other paths

as such, as soon as trees of conftests that use pytest_plugins, we get a inconsistent dependency state of plugins

additionally some plugins dod epend on other plugins either optionally or directly

experiments and a larger discussion are needed to give this issue the required detail for concrete action

[goodboy]

@RonnyPfannschmidt wait doesn't #52 provide a way to address this?

We can temporarily register contest.py plugins for each path - de-registering as each new path is used?

[RonnyPfannschmidt]

@tgoodlet #52 is absolutely dependent on this actually

after all temporary registrations imply sub-registrations of dependent plugins, and a context-manager is also not always desirable (in particular when plugins that add hooks come into play)

basically managing dependencies and dependent plugins is required in order to make something like the context manager sustainable

[goodboy]

as such, as soon as trees of conftests that use pytest_plugins, we get a inconsistent dependency state of plugins

@RonnyPfannschmidt do we have an example of this being a problem?
A bug in pytest maybe or some code as an example?
Is pytest-dev/pytest#1821 somewhat related?

additionally some plugins do depend on other plugins either optionally or directly

@RonnyPfannschmidt I was thinking about this a tiny bit.
The package management system really should take care of this?
Isn't it the plugin writers responsibility to ensure plugin dependencies are met (even if relying on other plugins)?
I agree that with the case of conftest.py files using pytest_plugins this isn't as explicit and works outside the pkg manager.

I guess I'm looking for a deeper explanation of the problem ;)

[RonnyPfannschmidt]

@tgoodlet the graph of plugin dependencies cant be drawn with the metadata python packaging provides atm

even more so if there is implicit and/or indirect dependence, currently we simply have no way to express the metadata needed to do it

[goodboy]

@RonnyPfannschmidt If you can give me an example of this problem it'd be super helpul ;)

we get a inconsistent dependency state of plugins

Mostly I just need to have an idea of what ^ consists of.

[RonnyPfannschmidt]

@tgoodlet any conftest that puts plugins into pytest_plugin enables them globally instead of for that folder tree

[pombredanne]

Hi: any progress on this?

[nicoddemus]

@pombredanne AFAIK nobody has taken the time to work on this, I'm afraid.

[pombredanne]

ack, thanks. I would need something like this, so I may hack a few things that may be of reuse. TBD!

[goodboy]

We always enjoy the help @pombredanne!

[pombredanne]

@tgoodlet I have something working mostly on my side that I will push in a branch of scancode-toolkit soon.

But in doing I found out that I may use only a few capabilities of pluggy. In particular I do not use pluggy to run the plugins, only as a registry. And my plugins are using Click for the CLI "UI".

The way dependencies are expressed is on other plugins is that each plugin holds a list of project_name: plugin_name strings where plugin_name is the setuptools entrypoint name. This is only an attribute of my plugins, not pluggy. Based onthis and the actual execution context of user-selected options:

1. I can check if a plugin is enabled (plugin-specific, typically based on its user-selected options or else)
2. If yes, I can check if all its dependencies are also enabled, recursively, by walking the dependencies graph, detecting and erroring on cycles, and checking if each plugin is enabled.
3. If yes, the plugin can run, otherwise an error is raised as dependencies are not met.

So all in all, it may be of little value for pluggy and pytest. And frankly I have no idea on how dependencies are handled by pluggy, though I would be glad to use this.

Anyway, I will ping you here when I push this.

[goodboy]

@pombredanne sounds good.
I'm sure it will at the least provide ideas for what @RonnyPfannschmidt desires here.

[pombredanne]

So I am making some slow progress aboutcode-org/scancode-toolkit#885 in a branch. I have now --with great shame-- realized that I am only using pluggy as a glorified setuptools entrypoint loader and do not use any of the hooking execution mechanisms.

My plugins are classes and not functions :|

Nevertheless I need to resolve deps between plugins and each of my plugins will need some try fist and try last and this with deps creates some graph.
For now I crudely do it at the UI level using an enhanced Click option class that can express dep requirements on other options for a plugin to be "enabled". I will need to build a better graph that's not UI dependent instead.

So here is my question 1.

• In evolving this mess o mine, could I use pluggy hooks to use classes instead? (I guess pluggy could work with any callable and not just functions. I need some state in each of my plugins, at least class-level state on a given "plugin")

And 2:

• If I can move there, the deps of my plugins could become regular pluggy deps (and I could get some of my code being of some help for this ticket). How do I express explicit and implicit deps between plugins in pluggy today?

[RonnyPfannschmidt]

@pombredanne

1. a pluggy plugin can be a class instance, so the state can act on multiple hooks
2. not at all