Adding interaction states for headless widgets. #19238

viridia · 2025-05-16T15:38:26Z

Part of #19236

This PR contains struct definitions for the low-level interaction states used by the headless widgets.

Part of bevyengine#19236

janhohenheim · 2025-05-17T17:25:17Z

Triage: failing tests

github-actions · 2025-05-19T17:47:06Z

It looks like your PR has been selected for a highlight in the next release blog post, but you didn't provide a release note.

Please review the instructions for writing release notes, then expand or revise the content in the release notes directory to showcase your changes.

ickshonpe · 2025-05-19T18:18:54Z

crates/bevy_picking/src/hover.rs

+/// and linear in the number of entities that have the `Hovering` component inserted.
+#[derive(Component, Copy, Clone, Default, Eq, PartialEq, Debug, Reflect)]
+#[reflect(Component, Default, PartialEq, Debug, Clone)]
+pub struct Hovering(pub bool);


The name Hovering seems a bit odd, isn't Hovered more idiomatic? It doesn't fit with Checked and ButtonPressed.

I have thought about this. In this context, the two words mean the same thing. Actually it's a bit more subtle than that: the -ed suffix has two interpretations in English, "checked" could mean "the past tense of the verb 'check'" (meaning that at some point in the past it was checked) or it could mean "currently in a state of checkness". "Hovering", OTOH, only has a single interpretation.

In any case, I don't see a clear winner between the two, and don't feel super strongly about it.

It seems wrong to me I think because the component belongs to the UI node entity and the node itself isn't Hovering, it's being Hovered by the pointer.

ickshonpe · 2025-05-19T18:37:32Z

crates/bevy_picking/src/hover.rs

+        return;
+    }
+
+    let Some(hover_map) = hover_map else { return };


This unwrap is probably cheaper than the is_empty check, so should be done first?

ickshonpe · 2025-05-19T18:59:22Z

crates/bevy_picking/src/hover.rs

+    // cases.
+    let mut hover_ancestors = EntityHashSet::with_capacity(32);
+    if let Some(map) = hover_map.get(&PointerId::Mouse) {
+        for (hovered_entity, _) in map.iter() {


Suggested change

for (hovered_entity, _) in map.iter() {

for hovered_entity in map.keys() {

ickshonpe · 2025-05-19T19:19:16Z

crates/bevy_picking/src/hover.rs

+    let mut hover_ancestors = EntityHashSet::with_capacity(32);
+    if let Some(map) = hover_map.get(&PointerId::Mouse) {
+        for (hovered_entity, _) in map.iter() {
+            hover_ancestors.insert(*hovered_entity);
+            hover_ancestors.extend(parent_query.iter_ancestors(*hovered_entity));
+        }
+    }
+
+    // For each hovered entity, it is considered "hovering" if it's in the set of hovered ancestors.
+    for (entity, mut hoverable) in hovers.iter_mut() {
+        let is_hovering = hover_ancestors.contains(&entity);
+        hoverable.set_if_neq(Hovering(is_hovering));
+    }


If no hover map is found or it's empty, then we can just set all the flags to false I think:

Suggested change

let mut hover_ancestors = EntityHashSet::with_capacity(32);

if let Some(map) = hover_map.get(&PointerId::Mouse) {

for (hovered_entity, _) in map.iter() {

hover_ancestors.insert(*hovered_entity);

hover_ancestors.extend(parent_query.iter_ancestors(*hovered_entity));

}

}

// For each hovered entity, it is considered "hovering" if it's in the set of hovered ancestors.

for (entity, mut hoverable) in hovers.iter_mut() {

let is_hovering = hover_ancestors.contains(&entity);

hoverable.set_if_neq(Hovering(is_hovering));

}

if let Some(map) = hover_map

.get(&PointerId::Mouse)

.filter(|map| !map.is_empty())

{

// Set which contains the hovered for the current pointer entity and its ancestors. The capacity

// is based on the likely tree depth of the hierarchy, which is typically greater for UI (because

// of layout issues) than for 3D scenes. A depth of 32 is a reasonable upper bound for most use

// cases.

let mut hover_ancestors = EntityHashSet::with_capacity(32);

for hovered_entity in map.keys() {

hover_ancestors.insert(*hovered_entity);

hover_ancestors.extend(parent_query.iter_ancestors(*hovered_entity));

}

// For each hovered entity, it is considered "hovering" if it's in the set of hovered ancestors.

for (entity, mut hoverable) in hovers.iter_mut() {

let is_hovering = hover_ancestors.contains(&entity);

hoverable.set_if_neq(Hovering(is_hovering));

}

} else {

for (_, mut hoverable) in hovers.iter_mut() {

hoverable.set_if_neq(Hovering(false));

}

}

If the hover map is empty, then the set of ancestors will be empty as well, which means that the flags will get set to false naturally. Note that in practice the hover map will rarely be empty, unless the mouse is outside the window bounds, or there are no hoverable entities under the pointer.

If there's no hover map at all - well. This is kinda sorta undefined behavior. I mean, it shouldn't crash. But there's really no point in having Hovering components if you haven't initialized the picking system.

That being said, if you really feel strongly about it, I could change it.

No don't worry about it, I'm being too fussy. It's fine as it is.

ickshonpe · 2025-05-19T19:38:01Z

crates/bevy_picking/src/hover.rs

+            hover_ancestors.extend(parent_query.iter_ancestors(*hovered_entity));
+        }
+    }
+
+    // For each hovered entity, it is considered "hovering" if it's in the set of hovered ancestors.
+    for (entity, mut hoverable) in hovers.iter_mut() {
+        let is_hovering = hover_ancestors.contains(&entity);
+        hoverable.set_if_neq(Hovering(is_hovering));
+    }


This seems wrong, hovered state isn't strictly transitive from child to parent since a child node can be positioned outside of its parents bounds.

I believe this is how the CSS :hover selector actually works: when you are hovering over a child element, you are technically also hovering over it's parent. I've tried to match that behavior, as far as I understand it.

Note that in CSS, the overflow property of the parent can affect whether the child is hoverable. However, the picking system should already handle this.

Yeah thinking about it there are circumstances where it makes sense that way.

Added release notes.

cart · 2025-05-21T15:11:39Z

crates/bevy_picking/src/hover.rs

+/// [`Interaction`]: https://docs.rs/bevy/0.15.0/bevy/prelude/enum.Interaction.html
+#[derive(Component, Copy, Clone, Default, Eq, PartialEq, Debug, Reflect)]
+#[reflect(Component, Default, PartialEq, Debug, Clone)]
+pub struct Hovered(pub bool);


I find it slightly confusing that the "hovered" state in Hovered is not the same as the "hovered" entity state in HoverMap. Ex: an entity not under a pointer with a child that is under a pointer is not "hovered" according to the HoverMap, but is "hovered" according to Hovered. Given that HoverMap is a public contract, I don't think we should be giving two different answers to the "what is hovered" question.

Well, I need something that replicates the behavior of CSS :hover pseudo-class. We can rename it if that would make things clearer.

Consider an HTML checkbox: even though the actual checkbox is a small widget, hovering or clicking on the label element is considered the same as clicking on the actual checkbox. Now, there are several ways to arrange this, for example you could have a parent element which includes both the checkbox and the label. But you could also make the label a child of the checkbox.

Similarly, one might envision a slider where the circular thumb element extends outside of the bounds of the track.

More broadly, the purpose of a visual hover indicator is to telegraph what will happen with the subsequent click - but since clicks bubble, a click on a child element, if not intercepted, will also be handled by the parent. The hover indicator needs to signal this. This is why the CSS :hover pseudo-class behaves as it does.

I also missed PickingInteraction::Hovered, which mirrors the state of HoverMap to a given entity. I think we need to come up with different terminology on one side or the other, as I consider this all to be a "single API surface".

There are now currently two "hover" concepts in bevy_picking:

"Cursor is directly over an unblocked Pickable::hover entity" (currently PickingInteraction::Hovered and HoverMap). This can be more than one entity, provided Pickable::should_block_lower is false for the first hovered entity highest in the stack.

"The result of (1) propagated up to all ancestors" (currently Hovered). Notably this currently ignores both Pickable::is_hoverable, Pickable::should_block_lower, as well as generally ignoring whether or not an ancestor is "pickable" at all in the first place (ex: any non-UI ancestor entity can still be Hovered, provided it has the component)

I don't have a solid solution off the top of my head, but I think its pretty clear this needs some additional consideration:

Do we need both concepts?

If so, how do we disambiguate these concepts?

Should we indiscriminately propagate Hovered up to all ancestors, ignoring whether or not they are pickable, whether or not they are "hoverable", etc?

I do think it makes sense to ignore the should_block_lower behavior in the context of Hovered.

Suppose I change the name of the component to something like HoverWithin?

I think that should_block_lower is orthogonal here: it controls whether an entity is pickable, but an entity can be non-pickable and still "contain" a hovered entity, that is, the "within" predicate is independent of whether or not the entity itself is pickable. And the non-pickable entity will still get a bubbled click event from its child, so telegraphing of future clicks should be consistent.

In terms of use cases: "hover within" is the more common case, at least in my experience (YMMV). Complex widgets which are aggregates of multiple entities generally fall into two groups:

Widgets which are artistic composites of parts, but which are singular in interaction (e.g. a button containing a text label and an icon). For these cases, "hover within" is the right predicate.

Widgets which have multiple "handles" or "cells" that are independently draggable or selectable. In this case, each of the individual handle entities will likely have it's own Hover component and highlight state.

It is possible, however, to envision a widget which doesn't fall into either scheme - for example, a widget with a large number of handles or cells, where the hovering behavior is handled at the parent level via some kind of multiplexing scheme. This widget would want to know exactly which child was hovered, however this gets tricky because the child may itself have children (text content) and so you end up having to do some kind of ancestor check anyway. I'm having a hard time envisioning a concrete use case for this, but it's not impossible.

Here's another anecdote: A classic novice error in Bevy UI is where you have a button with a text label, and you implement a hover effect when the button is hovered - only to discover that when the mouse hovers over the text the button is no longer highlighted, because now the text, rather than the button, is in the hover map. Then what happens is that you ask on discord (this has happened) and what people tell you is to make the text non-pickable. This is the wrong answer.

It's wrong because buttons shouldn't impose this constraint on their contents. In a BSN world, buttons may contain text, images, or complex hierarchies of entities, because the content of a button is a template parameter. It's onerous to make the user sanitize all of this content to ensure that none of the entities being passed into the button are pickable.

Then what happens is that you ask on discord (this has happened) and what people tell you is to make the text non-pickable. This is the wrong answer.

Agreed. I do think the "hovered within" approach is the better solve here.

So "answering" my questions one by one:

Do we need both concepts?

I'm reasonably convinced that we need both. The "pointer hover" system feeds into "hovered within", and I can imagine use cases that care about direct hovering info.

If so, how do we disambiguate these concepts?

We don't currently have events or components for each concept, but I think we should consider these all "peers" of each other:

pointer over: there is a pointer directly above this entity (note that it looks like the Pointer<Over> event is driven using the HoverMap not the OverMap. We might need to rename that? Also see Pointer<Enter> and Pointer<Leave> #16289)

hovered: there is a pointer over this entity, and no entity "above" it in the stack is blocking it

hovered within: either this entity is hovered, or one of its descendants is

I think "hovered" and "hovered within" work reasonably well. Although "hovered within" is less friendly (and less immediately clear) than "hovered". In general I think people will try to reach for the "hovered" name over "hovered within", then learn that what they actually want is "hovered within".

Working backwards, if we want the "hovered within" concept to be known as "hovered" (to bias people toward using that), what can we call the "pointer hover" concept to disambiguate? Perhaps "pointer hover" vs "hover" is suitably clear, provided we always specify "pointer hover" in that context (ex: PointerHoverMap, Pointer::Hover, etc).

It is worth pulling in @NthTensor and @aevyrie to see what they think here.

Should we indiscriminately propagate Hovered up to all ancestors, ignoring whether or not they are pickable, whether or not they are "hoverable", etc?

Ok yeah I see the value here. Ex: this system will also be used outside of UI, and scenes will often have a simple Transform-only entity at the top, which is not directly "pickable" by a backend. That scene could easily want a top level hover event.

cart · 2025-05-21T15:16:34Z

crates/bevy_ui/src/interaction_states.rs

+
+// Hook to set the a11y "disabled" state when the widget is disabled.
+fn on_add_disabled(mut world: DeferredWorld, context: HookContext) {
+    let mut entt = world.entity_mut(context.entity);


Nit: entt seems like a slightly awkward name for this.

BTW the name "entt" (pronounced "en-tee-tee") is one of the "short local variable names" that I often use in my code, I picked up the name from some Bevy code example, don't remember where.

cart · 2025-05-21T15:20:52Z

crates/bevy_ui/src/interaction_states.rs

+/// Component that indicates whether a button is currently pressed. This will be true while
+/// a drag action is in progress.
+#[derive(Component, Default, Debug)]
+pub struct ButtonPressed(pub bool);


Given that the other "bool state" components are immutable, maybe we should make this immutable too for consistency / to enable reliable observer-driven management?

cart · 2025-05-21T15:33:39Z

crates/bevy_ui/src/interaction_states.rs

+    }
+}
+
+/// Component that indicates whether a button is currently pressed. This will be true while


There is nothing enforcing this behavior at the moment. If this is a "universal" state intended to be shared and not something specific to, say, the upcoming "bevy core widgets" (which is where this behavior is implemented in your prototype), I think it should call out explicitly that this is a contract that implementers are expected to fulfill.

That being said, maybe whether or not a button is pressed on drag should be a widget-specific behavior? (aka maybe we should delete this line, then document the drag behavior in the relevant core widget docs?)

Also note that we can't call this Pressed because that name is already used in the prelude.

It's an open question how universal we want this to be. Some widgets are pressable and others are not. But "pressing" has uses outside of UI - for example PickingInteraction lives in bevy_picking, not bevy_ui, and has a Pressed variant.

Yeah this does feel a little awkward as we already have a universal "pressing" system (bevy_picking::Pressed). But by adopting the "controlled" model, we are decoupling the state of the button from that. I think this is fine: it is the job of the developer to "rewire" the bevy picking state to the button state, if they so choose. These systems are compatible / harmonious with each other. I wouldn't want to call this Pressed even if that name wasn't in the prelude, as that creates a situation where something is "pressed" according to bevy_picking, but "not pressed" according to the local UI state. At least with ButtonPressed we put a little bit of space between the two concepts (although maybe using different terms like Activated would disambiguate appropriately) .

Part of me does want to embrace an "uncontrolled" model here, as that would allow us to more directly unify the bevy_picking concepts/events/components with bevy_ui. When considering both this case and the hovered case, it feels like we aren't building a cohesive framework of concepts. Users will be confused.

Nit: Buttons are neither controlled or uncontrolled because they don't have values - that is, while buttons do have state (hover and pressed), that state is meant for internal use only. Yes, the button state is accessible to allow for custom visual styling, but that state is not part of the protocol by which the button communicates with the rest of the app - it's not like a slider or checkbox which controls a value.

What this means is that controlled widgets are only "controlled" with respect to the primary value being edited. All other state variables (like drag mode and offset) are managed by the widget internally, without the need for cooperation from the app. Or to put it even simpler: classic MVC.

Ok yeah that makes sense. I was imagining the pressed state as similar to the "checked" state, in that a developer might want to gate whether or not it is actually "pressed" based on their own criteria. But that seems less likely.

Clarified comments. Made ButtonPressed immutable.

Added IsHoveredDirect. Added additional tests for IsHoveredDirect. Added `get` accessor to `Hovered` and `IsHoveredDirect`.

cart

I like this direction and think this is pretty much good to merge, but I think we should wait until we have the other concept renames we discussed queued up in a separate PR (with approvals / consensus from the relevant people), as otherwise we're taking on "conceptual dept" on main. The concept naming in picking is already hard to follow (prior to this PR), and the multiple conflicting "hovered" concepts isn't something I want sitting on main for any period of time.

cart · 2025-05-23T00:36:01Z

crates/bevy_picking/src/hover.rs

+/// The component's boolean value will be `true` whenever the pointer is currently hovering over the
+/// entity, or any of the entity's children. This is consistent with the behavior of the CSS
+/// `:hover` pseudo-class, which applies to the element and all of its children.


I think it makes sense to say "descendants" here instead of "children" as this applies to arbitrary levels of the hierarchy. It would also be good to use "directly" here, to connect this to HoveredDirect.

Suggested change

/// The component's boolean value will be `true` whenever the pointer is currently hovering over the

/// entity, or any of the entity's children. This is consistent with the behavior of the CSS

/// `:hover` pseudo-class, which applies to the element and all of its children.

/// The component's boolean value will be `true` whenever the pointer is currently directly hovering over the

/// entity, or any of the entity's descendants (as defined by the [`ChildOf`] relationship). This is consistent with the behavior of the CSS

/// `:hover` pseudo-class, which applies to the element and all of its children.

cart · 2025-05-23T00:37:48Z

crates/bevy_picking/src/hover.rs

+/// hover state.
+#[derive(Component, Copy, Clone, Default, Eq, PartialEq, Debug, Reflect)]
+#[reflect(Component, Default, PartialEq, Debug, Clone)]
+pub struct IsHoveredDirect(pub bool);


I think "directly hovered" feels more natural than "hovered direct". Thoughts?

cart · 2025-05-23T00:41:16Z

crates/bevy_picking/src/hover.rs

+/// enters or leaves an entity. Users should insert this component on an entity to indicate interest
+/// in knowing about hover state changes.
+///
+/// This is similar to the old Bevy [`Interaction`] component, except that it only tracks


This feels a bit awkward as "old" implies Interaction either no longer exists or is deprecated. Otherwise this is just a history lesson about what came first, which isn't really something users need to care about.

ickshonpe · 2025-05-23T10:40:57Z

crates/bevy_picking/src/hover.rs

+/// [`Interaction`]: https://docs.rs/bevy/0.15.0/bevy/prelude/enum.Interaction.html
+#[derive(Component, Copy, Clone, Default, Eq, PartialEq, Debug, Reflect)]
+#[reflect(Component, Default, PartialEq, Debug, Clone)]
+pub struct IsHovered(pub bool);


Possibly this naming could be confusing because Is- components in bevy tend to be unit structs used as markers? It seems natural to use a With<IsHovered> query filter assuming that it would return only hovered UI nodes.

Adding interaction states for headless widgets.

5e153ed

Part of bevyengine#19236

janhohenheim added the S-Needs-Review Needs reviewer attention (from anyone!) to move forward label May 17, 2025

janhohenheim added A-UI Graphical user interfaces, styles, layouts, and widgets S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged and removed S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels May 17, 2025

viridia added 2 commits May 19, 2025 10:42

Moved Hovering to bevy_picking. Added tests.

743ca0d

Spelling error.

cdc2b04

janhohenheim added S-Needs-Review Needs reviewer attention (from anyone!) to move forward and removed S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged labels May 19, 2025

alice-i-cecile added C-Feature A new feature, making something new possible M-Needs-Release-Note Work that should be called out in the blog due to impact labels May 19, 2025

alice-i-cecile added this to Alice's Work Planning May 19, 2025

alice-i-cecile moved this to Widget-ready in Alice's Work Planning May 19, 2025

viridia added 2 commits May 19, 2025 11:55

Add release notes.

508fffd

Spelling

a98547b

ickshonpe reviewed May 19, 2025

View reviewed changes

ickshonpe suggested changes May 19, 2025

View reviewed changes

viridia added 6 commits May 20, 2025 06:56

Added changes from review comments.

11aaf8a

Added release notes.

Updated comment.

6e4fc7f

Fixed some doc links.

58fa91a

Hovering -> Hovered

50f0da6

Revert over-zealous change.

48c98d9

Another revert.

c38af19

cart added this to @cart's attention May 20, 2025

cart moved this to Respond (With Priority) in @cart's attention May 20, 2025

ickshonpe approved these changes May 21, 2025

View reviewed changes

cart reviewed May 21, 2025

View reviewed changes

Renamed local vars.

090307a

Clarified comments. Made ButtonPressed immutable.

viridia added 2 commits May 21, 2025 13:17

Merge remote-tracking branch 'upstream/main'

5547e8e

Added comment explaining the algorithm.

25d9d44

cart requested review from aevyrie and NthTensor May 22, 2025 21:22

Changed Hovered to IsHovered.

fc3943f

Added IsHoveredDirect. Added additional tests for IsHoveredDirect. Added `get` accessor to `Hovered` and `IsHoveredDirect`.

cart reviewed May 23, 2025

View reviewed changes

ickshonpe reviewed May 23, 2025

View reviewed changes

Renamings and comment updates based on review feedback.

7d83ebc

	for (hovered_entity, _) in map.iter() {
	for hovered_entity in map.keys() {

Uh oh!

Adding interaction states for headless widgets. #19238

Are you sure you want to change the base?

Adding interaction states for headless widgets. #19238

Conversation

viridia commented May 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

janhohenheim commented May 17, 2025

Uh oh!

github-actions bot commented May 19, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ickshonpe May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

viridia commented May 16, 2025 •

edited

Loading

ickshonpe May 19, 2025 •

edited

Loading

ickshonpe May 23, 2025 •

edited

Loading