Nested Queries #21557

chescock · 2025-10-15T15:06:18Z

Objective

Support queries that soundly access multiple entities.

This can be used to create queries that follow relations, as in #17647.

This can also be used to create queries that perform resource access. This has been supported since #16843, although that approach may become unsound if we do resources-as-components #19731, such as #21346.

Fixes #20315

Solution

Allow a QueryData that wants to access other entities to store a QueryState<D, F> in its WorldQuery::State, so that it can create a nested Query<D, F> during the outer fetch.

New `WorldQuery` methods

For it to be sound to create the Query during fetch, we need to register the FilteredAccess of the nested query and check for conflicts with other parameters. Create a WorldQuery::update_external_component_access method for that purpose. For Query as SystemParam, call this during init_access so the access can be combined with the rest of the system access. For loose QueryStates, call it during QueryState::new.

In order to keep the query cache up-to-date, create a WorldQuery::update_archetypes method where it can call QueryState::update_archetypes_unsafe_world_cell, and call it from there.

New `QueryData` subtraits

Some operations would not be sound with nested queries! In particular, we want a Parent<D> query that reads data from the parent entity by following the ChildOf relation. But many entities may share a parent, so it's not sound to iterate a Query<Parent<&mut C>>.

It is sound to get_mut, though, so we want the query type to exist, just not be iterable. And following the relation in the other direction for a Query<Children<&mut C>> is sound to iterate, since children are unique to a given parent.

So, introduce two new QueryData subtraits:

IterQueryData - For anything it's sound to iterate. This is used to bound iter_mut and related methods.
SingleEntityQueryData - For anything that only accesses data from one entity. This is used to bound EntityRef::get_components (see Unsound to call EntityRef::get_components with a QueryData that performs resource access #20315). It's also used to bound transmute at the moment, although we may be able to relax that later (see below, under Future Work).

Note that SingleEntityQueryData: IterQueryData, since single-entity queries never alias data across entities, and ReadOnlyQueryData: IterQueryData, since it's always sound to alias read-only data.

Here is a summary of the traits implemented by some representative QueryData:

Data	Iter	ReadOnly	SingleEntity
`&T`	✓	✓	✓
`&mut T`	✓	x	✓
`Parent<&T>`	✓	✓	x
`Parent<&mut T>`	x	x	x
`(&mut T, Parent<&U>)`	✓	x	x
`Children<&mut T>`	✓	x	x

Alternatives

We could avoid the need for the IterQueryData trait by making it a requirement for all QueryData. That would reduce the number of traits required, at the cost of making it impossible to support Query<Parent<&mut C>>.

Showcase

Here is an implementation of a Related<R, D, F> query using this PR that almost works:

struct Related<R: Relationship, D: QueryData + 'static, F: QueryFilter + 'static>(PhantomData<(R, D, F)>);

struct RelatedState<R: Relationship, D: QueryData + 'static, F: QueryFilter + 'static> {
    relationship: <&'static R as WorldQuery>::State,
    nested: QueryState<D, (F, With<R::RelationshipTarget>)>,
}

struct RelatedFetch<'w, R: Relationship> {
    relationship: <&'static R as WorldQuery>::Fetch<'w>,
    world: UnsafeWorldCell<'w>,
    last_run: Tick,
    this_run: Tick,
}

impl<R: Relationship> Clone for RelatedFetch<'_, R> {
    fn clone(&self) -> Self {
        Self { relationship: self.relationship.clone(), world: self.world, last_run: self.last_run, this_run: self.this_run }
    }
}

unsafe impl<R: Relationship, D: QueryData + 'static, F: QueryFilter + 'static> WorldQuery
    for Related<R, D, F>
{
    type Fetch<'w> = RelatedFetch<'w, R>;
    type State = RelatedState<R, D, F>;

    fn shrink_fetch<'wlong: 'wshort, 'wshort>(fetch: Self::Fetch<'wlong>) -> Self::Fetch<'wshort> {
        let RelatedFetch { relationship, world, last_run, this_run } = fetch;
        let relationship = <&R>::shrink_fetch(relationship);
        RelatedFetch { relationship, world, last_run, this_run }
    }

    unsafe fn init_fetch<'w, 's>(world: UnsafeWorldCell<'w>, state: &'s Self::State, last_run: Tick, this_run: Tick) -> Self::Fetch<'w> {
        let relationship = unsafe { <&R>::init_fetch(world, &state.relationship, last_run, this_run) };
        RelatedFetch { relationship, world, this_run, last_run }
    }

    const IS_DENSE: bool = <&R>::IS_DENSE;

    unsafe fn set_archetype<'w, 's>(fetch: &mut Self::Fetch<'w>, state: &'s Self::State, archetype: &'w Archetype, table: &'w Table) {
        unsafe { <&R>::set_archetype(&mut fetch.relationship, &state.relationship, archetype, table) };
    }

    unsafe fn set_table<'w, 's>(fetch: &mut Self::Fetch<'w>, state: &'s Self::State, table: &'w Table) {
        unsafe { <&R>::set_table(&mut fetch.relationship, &state.relationship, table) };
    }

    fn update_component_access(state: &Self::State, access: &mut FilteredAccess) {
        <&R>::update_component_access(&state.relationship, access);
    }

    fn update_external_component_access(state: &Self::State, system_name: Option<&str>, component_access_set: &mut FilteredAccessSet, world: UnsafeWorldCell) {
        state.nested.update_external_component_access(system_name, component_access_set, world);
    }

    fn init_state(world: &mut World) -> Self::State {
        RelatedState {
            relationship: <&R>::init_state(world),
            nested: unsafe { QueryState::new_unchecked(world) },
        }
    }

    fn get_state(_components: &Components) -> Option<Self::State> {
        // :(
        None
    }

    fn matches_component_set(state: &Self::State, set_contains_id: &impl Fn(ComponentId) -> bool) -> bool {
        <&R>::matches_component_set(&state.relationship, set_contains_id)
    }

    fn update_archetypes(state: &mut Self::State, world: UnsafeWorldCell) {
        state.nested.update_archetypes_unsafe_world_cell(world);
    }
}

unsafe impl<R: Relationship, D: QueryData + 'static, F: QueryFilter + 'static> QueryData
    for Related<R, D, F>
{
    const IS_READ_ONLY: bool = D::IS_READ_ONLY;
    type ReadOnly = Related<R, D::ReadOnly, F>;
    type Item<'w, 's> = Option<D::Item<'w, 's>>;

    fn shrink<'wlong: 'wshort, 'wshort, 's>(item: Self::Item<'wlong, 's>) -> Self::Item<'wshort, 's> {
        item.map(D::shrink)
    }

    unsafe fn fetch<'w, 's>(state: &'s Self::State, fetch: &mut Self::Fetch<'w>, entity: Entity, table_row: TableRow) -> Self::Item<'w, 's> {
        let relationship = <&R>::fetch(&state.relationship, &mut fetch.relationship, entity, table_row);
        let query = unsafe { state.nested.query_unchecked_manual_with_ticks(fetch.world, fetch.last_run, fetch.this_run ) };
        query.get_inner(relationship.get()).ok()
    }
}

unsafe impl<R: Relationship, D: ReadOnlyQueryData + 'static, F: QueryFilter + 'static> ReadOnlyQueryData for Related<R, D, F> { }

// Note that we require `D: ReadOnlyQueryData` for `Related: IterQueryData`
unsafe impl<R: Relationship, D: ReadOnlyQueryData + 'static, F: QueryFilter + 'static> IterQueryData for Related<R, D, F> { }

That has a few flaws, notably that it fails with

error[E0271]: type mismatch resolving `<Related<R, <D as QueryData>::ReadOnly, F> as WorldQuery>::State == RelatedState<R, D, F>`

because QueryData requires that State = ReadOnly::State, but QueryState<D, F> != QueryState<D::ReadOnly, F>.

It's also impossible to implement get_state, because constructing a QueryState requires reading the DefaultQueryFilters resource, but get_state can be called from transmute with no access.

I believe it's possible to resolve those issues, but I don't think those solutions belong in this PR.

Future Work

There is more to do here, but this PR is already pretty big. Future work includes:

WorldQuery types for working with relationships #17647
Following Store resources as components on singleton entities (v2) #21346, update AssetChanged to use nested queries for resource access, and stop tracking resource access separately in Access
Relax the SingleEntityQueryData bound on transmutes and joins. This will require checking that the nested query access is also a subset of the original access. Although unless we also solve the problem of get_state being impossible to implement, transmuting to a query with nested queries won't work anyway.
Support streaming iteration for QueryIter by offering a fn fetch_next(&self) -> D::Item<'_> method and relaxing the IterQueryData bound on Query::into_iter and Query::iter_mut. This would work similar to iter_many_mut and iter_many_inner.
Relax the IterQueryData bound on Query::single_inner, Query::single_mut, and Single<D, F>. This seems like it should be straightforward, because the method only returns a single item. But the way it checks that there is only one item is by fetching the second one!

This should be possible, but the access checks are tricky.

…rom nested queries.

Freyja-moth · 2025-10-15T21:05:23Z

so it's not sound to iterate a Query<Parent<&mut C>>.

I'm not certain it's the best way of going about it but couldn't we implement this by storing the entity of the parent instead of the data and then resolving the data from the entity when it is needed?

cBournhonesque · 2025-10-16T01:09:41Z

crates/bevy_ecs/src/system/query.rs

-    pub fn iter_mut(&mut self) -> QueryIter<'_, 's, D, F> {
+    pub fn iter_mut(&mut self) -> QueryIter<'_, 's, D, F>
+    where
+        D: IterQueryData,


The PR looks good, but I don't fully understand IterQueryData.

The trait says

A [`QueryData`] for which instances may be alive for different entities concurrently.

But when doing iter_mut, we never have two 2 items concurrently right? since we would iterate through them one by one. So even if we have a nested query which accesses multiple entities, there would still not be a collision?

But when doing iter_mut, we never have two 2 items concurrently right? since we would iterate through them one by one.

Nope, you can keep the old items around! The lifetime in fn next(&mut self) isn't connected to the Item type, so later calls don't invalidate earlier items. That's how things like collect() work.

Nope, you can keep the old items around! The lifetime in fn next(&mut self) isn't connected to the Item type, so later calls don't invalidate earlier items. That's how things like collect() work.

This is a really helpful explanation. Can you add something like this to the IterQueryData docs?

cBournhonesque · 2025-10-16T01:10:42Z

crates/bevy_ecs/src/system/query.rs

    /// ```
    #[track_caller]
-    pub fn transmute_lens<NewD: QueryData>(&mut self) -> QueryLens<'_, NewD> {
+    pub fn transmute_lens<NewD: SingleEntityQueryData>(&mut self) -> QueryLens<'_, NewD> {


How come we can only transmute to a SingleEntityQueryData?

Ah i see in the PR description that this could be relaxed in the future

chescock · 2025-10-16T14:18:08Z

couldn't we implement this by storing the entity of the parent instead of the data and then resolving the data from the entity when it is needed?

I'm not sure what you mean. Like, instead of yielding a D::Item<'w, 's>, we could yield some type Foo with a fn get_mut(&mut self) -> D::Item<'_, '_>? That wouldn't help, since it would still be possible to collect the Foo values and then call get_mut on several of them concurrently, like:

let mut items = query.iter_mut().collect::<Vec<_>>();
let mapped = items.iter_mut().map(|item| item.get_mut()).collect::<Vec<_>>();

cBournhonesque · 2025-10-18T18:05:52Z

crates/bevy_ecs/src/query/state.rs

    }

+    /// Collects the access from this query and any nested queries
+    /// and panics


Was this doc cut short? This seems to be doing way more than simply collecting queries; it basically replaces the old job of <Query as SystemParam>::init_access where we would check if the Query conflicts with other SystemParams of the System.

Was this doc cut short?

Oh, yup, I'll fix that! I must have accidentally a

This seems to be doing way more than simply collecting queries; it basically replaces the old job of <Query as SystemParam>::init_access where we would check if the Query conflicts with other SystemParams of the System.

Yup, it's the same as init_access; it collects the FilteredAccesses into a FilteredAccessSet and panics if there are conflicts.

Hmm, I should really rename it. I think in an earlier draft it just handled the access from nested queries, but now it also handles the access from the current query, so "external" is a misleading name. I'll try QueryState::init_access to make it more clear that it's the same as SystemParam::init_access, and then rename WorldQuery::update_external_component_access to WorldQuery::init_nested_access.

cBournhonesque · 2025-10-18T19:06:05Z

crates/bevy_ecs/src/query/state.rs

+        }
+
+        component_access_set.add(self.component_access.clone());
+        D::update_external_component_access(


Wouldn't we want to collect the access from nested queries before checking if they cause conflicts with other SystemParams? Or would the conflict become apparent later?

It would be nice to have at least one implementation of a QueryData that implements update_external_component_access so we can add unit tests, but I understand that the PR is already big

Ah is the answer that D::update_external_component_access must itself check for conflicts and possibly panic?

But in that case couldn't we just update self.component_access with the nested component accesses?
Where is the distinction between "current entity component access" and "nested entity component access" relevant?

Wouldn't we want to collect the access from nested queries before checking if they cause conflicts with other SystemParams? Or would the conflict become apparent later?

I think either a preorder or postorder traversal would work. This is essentially visiting all of the Query instances and calling init_access on each one, and it doesn't matter whether we process the outer query before or after the nested ones, since the conflict check is symmetric.

It would be nice to have at least one implementation of a QueryData that implements update_external_component_access so we can add unit tests, but I understand that the PR is already big

Yeah, sorry. I wrote out an example impl for the PR description, but there's a little more prep work before we can commit a complete implementation, and creating a temporary one just for unit tests didn't seem worthwhile.

But in that case couldn't we just update self.component_access with the nested component accesses? Where is the distinction between "current entity component access" and "nested entity component access" relevant?

That's what I tried at first! But the query infrastructure also uses the filters in component_access to determine which archetypes match the query, so you need to keep track of the "current entity" access for that purpose.

Then for a while I had both a FilteredAccess for the main query and a FilteredAccessSet for the nested queries. But that wound up storing multiple copies of each access, and it was easy enough to walk through the tree to collect them when needed.

It would be nice to have at least one implementation of a QueryData that implements update_external_component_access so we can add unit tests

Thinking about this further, I think I should add a generic NestedQuery implementation. It's not useful in application code, since you could just add another Query system parameter, but I think it will be useful to delegate to, either in manual QueryData impls or with #[derive(QueryData)]. And that would give us an actual implementation to write unit tests with!

I'll try to put that together on Monday.

cBournhonesque · 2025-10-18T19:28:11Z

I think these changes are a great idea; I guess I would like to know how this contrasts to how flecs queries for relation data. Maybe @james-j-obrien knows?

james-j-obrien · 2025-10-18T19:57:17Z

I think these changes are a great idea; I guess I would like to know how this contrasts to how flecs queries for relation data. Maybe @james-j-obrien knows?

That's a quite involved question to answer (although a very interesting one).

The big difference between the bevy and flecs queries are tied to one being defined in the type system and the other being defined dynamically. Due to this bevy queries are fundamentally based on nesting, you have tuples of query terms that each store their own state and generate their own code for managing that state. In flecs all the query terms are just stored in a flat array.

For example in this PR we express querying our parent as creating a query term that traverses the relationship and then nested in that is the set of components we want to access on the target, whereas in flecs you would have a set of instructions that said: "get me any entity A with relationship of the form (ChildOf, B) and store B as a variable", "get me component Y on entity B", "get me component Z on entity B".

This structure allows flecs to optimize/batch/reorder terms since they can be evaluated in the full context of the rest of the query, but for simple queries it's mostly a different path to the same goal.

Since flecs also has fragmenting relations they can do stuff like cache the tables for B since you know that entities in A's table will always have parent B.

All that being said, with bevy's queries as they exist today this PR seems like the shortest path to querying on multiple entities so seems like a positive step.

…` and fix missing documentation.

…:init_nested_access`.

cBournhonesque · 2025-10-19T13:17:08Z

I think these changes are a great idea; I guess I would like to know how this contrasts to how flecs queries for relation data. Maybe @james-j-obrien knows?

That's a quite involved question to answer (although a very interesting one).

The big difference between the bevy and flecs queries are tied to one being defined in the type system and the other being defined dynamically. Due to this bevy queries are fundamentally based on nesting, you have tuples of query terms that each store their own state and generate their own code for managing that state. In flecs all the query terms are just stored in a flat array.

For example in this PR we express querying our parent as creating a query term that traverses the relationship and then nested in that is the set of components we want to access on the target, whereas in flecs you would have a set of instructions that said: "get me any entity A with relationship of the form (ChildOf, B) and store B as a variable", "get me component Y on entity B", "get me component Z on entity B".

This structure allows flecs to optimize/batch/reorder terms since they can be evaluated in the full context of the rest of the query, but for simple queries it's mostly a different path to the same goal.

Since flecs also has fragmenting relations they can do stuff like cache the tables for B since you know that entities in A's table will always have parent B.

All that being said, with bevy's queries as they exist today this PR seems like the shortest path to querying on multiple entities so seems like a positive step.

Thanks for the answer!
I guess bevy can also use the dynamic QueryBuilder, but the entire design is heavily influenced by primarily using types.
After reading https://ajmmertens.medium.com/building-games-in-ecs-with-entity-relationships-657275ba2c6c, it sounds like flecs creates some kind of data structure (node graph) that allow it to efficiently match entities.

I guess we could do something similar: for tuple queries, build such a node graph and use it to match entities. I guess we do already create a data structure that helps us find matching entities; that data structure is the QueryState.
The main difference that our state simply has:

matched archetypes (from QueryData::matches_component_set)
uses types to filter out entities (F::filter)

And the main difference is that the flecs "QueryState" is more elaborate since it can contain sources, relationships, etc.
So this PR's NestedQueries is one way to add more complexity to our QueryState. But we still have a simple 'combined query' since our Tuple QueryState combines the inner WorldQueries' QueryState in a very simple manner. In flecs it would combine them by adding them into a dynamic graph that can then be optimized. Our equivalent would be to add a WorldQuery::add_to_query_plan method so that we would also be able to optimize a query that contains multiple terms

alice-i-cecile · 2025-10-19T19:50:09Z

@eugineerd, I liked your work over in #21601; can I get your review here in turn?

alice-i-cecile · 2025-10-20T04:03:20Z

crates/bevy_ecs/src/query/fetch.rs

+/// or that only perform read access on other entities.
+/// Queries may perform mutable access on other entities if they
+/// can prove that the other entities are all distinct,
+/// such as if they are on the appropriate side of a one-to-many or one-to-one relation.


Do we have the corresponding safety constraints on the relationship traits in place already? I think I agree that banning self-targeted / non-unique relations is correct, but we should at least have safety requirements there.

Do we need to swap our relations implementations to ban e.g. Vec in favor of the unique entity equivalents?

alice-i-cecile · 2025-10-20T04:06:36Z

crates/bevy_ecs/src/query/iter.rs

    /// # schedule.run(&mut world);
    /// ```
-    pub fn sort<L: ReadOnlyQueryData + 'w>(
+    pub fn sort<L: ReadOnlyQueryData + SingleEntityQueryData + 'w>(


Doc comments about why SingleEntityQueryData is required for these methods would be helpful for users. It's not immediately clear why this is required.

alice-i-cecile · 2025-10-20T04:09:13Z

crates/bevy_ecs/src/query/world_query.rs


+    /// Adds any component accesses to other entities used by this [`WorldQuery`].
+    ///
+    /// This method should panic if the access would conflict with any existing access in the [`FilteredAccessSet`].


Is this a "should panic" or a "must panic"?

alice-i-cecile · 2025-10-20T04:09:45Z

crates/bevy_ecs/src/query/world_query.rs

+    /// This method should panic if the access would conflict with any existing access in the [`FilteredAccessSet`].
+    ///
+    /// This is used for queries to request access to entities other than the current one,
+    /// such as to read resources or to follow relations.


Suggested change

/// such as to read resources or to follow relations.

/// such as to follow relations.

Resources as entities isn't merged, so we shouldn't add confusing doc comments.

chescock added 7 commits October 14, 2025 21:46

Extra traits for IterableQueryData and SingleEntityQueryData.

67d2a1b

Bound get_components with SingleEntityQueryData.

8083d79

Add IterQueryData bounds where necessary.

4ab98f3

Bound transmute and join with SingleEntityQueryData.

214a47f

This should be possible, but the access checks are tricky.

WorldQuery::update_archetypes to update caches for nested queries.

3a16fca

WorldQuery::update_external_component_access to accumulate access f…

be4c890

…rom nested queries.

Migration guide.

a5cffe6

chescock added 2 commits October 15, 2025 11:07

Update PR number in migration guide.

dcbe137

Lists should be surrounded by blank lines

19be88a

alice-i-cecile added D-Complex Quite challenging from either a design or technical perspective. Ask for help! M-Needs-Release-Note Work that should be called out in the blog due to impact labels Oct 15, 2025

alice-i-cecile added this to the 0.18 milestone Oct 15, 2025

cBournhonesque reviewed Oct 16, 2025

View reviewed changes

cBournhonesque reviewed Oct 18, 2025

View reviewed changes

chescock added 2 commits October 18, 2025 20:21

Rename QueryState::update_external_component_access to `init_access…

09d9a48

…` and fix missing documentation.

Rename WorldQuery::update_external_component_access to `WorldQuery:…

457663a

…:init_nested_access`.

alice-i-cecile requested a review from Trashtalk217 October 19, 2025 19:51

alice-i-cecile self-requested a review October 19, 2025 19:51

alice-i-cecile reviewed Oct 20, 2025

View reviewed changes

	/// such as to read resources or to follow relations.
	/// such as to follow relations.

Uh oh!

Nested Queries #21557

Are you sure you want to change the base?

Nested Queries #21557

Conversation

chescock commented Oct 15, 2025

Objective

Solution

New WorldQuery methods

New QueryData subtraits

Alternatives

Showcase

Future Work

Uh oh!

Freyja-moth commented Oct 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chescock commented Oct 16, 2025

Uh oh!

cBournhonesque Oct 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cBournhonesque commented Oct 18, 2025

Uh oh!

james-j-obrien commented Oct 18, 2025

Uh oh!

cBournhonesque commented Oct 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alice-i-cecile commented Oct 19, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

New `WorldQuery` methods

New `QueryData` subtraits

cBournhonesque Oct 18, 2025 •

edited

Loading

cBournhonesque commented Oct 19, 2025 •

edited

Loading