Gizmos rework

[lynn-lumen]

The following is based on the ideas from and issues encountered in

• Generic composite primitive gizmos / rework #21480,
• Gizmos as structs (and more !) #20246 and
• GizmoPrimitiveNd for external primitives #13615

Seeing that reworking gizmos has been controversial or contentious, I think that having a discussion about the approach we want to take here before implementing those changes is sensible.

This is an attempt at fixing the issues and enabling the features encountered and proposed there. Please feel free to leave feedback.

Issues / Features

The current implementation of gizmos for primitives is fairly flexible but has reached some limitations encountered in the PRs/issues mentioned above.
In particular, the following features could be added/issues could be fixed in a future rework:

• Extensibility: It should be possible to implement gizmos for primitives defined outside of bevy_math / bevy_gizmos.
• Composability: It should be possible to construct gizmos for two or more primitives at the same time, modify the gizmos and then draw them. This would be particularly useful for primitives that are composed of multiple constituent primitives like Extrusion or Ring.
• Component: We could define a GizmoComponent struct and implement Component for it. This component could draw a specified Gizmo after applying the entities Transform to the Gizmo. This could provide a uniform solution for drawing hitboxes or colliders.
• 2D primitives in 3D: Currently, 2D primitives can be drawn in 2D space using gizmos.primitive_2d(..) and only select 2D primitives like Ellipse or Rect can be drawn in 3D space using functions like gizmos.ellipse(..). We could allow drawing arbitrary 2D primitives in 3D space.
• Common application of Isometry: Drawing a primitive usually involves calculating vertex positions as if the primitive was not rotated/translated and transforming by an isometry later. This process could be unified across all primitives. The advantages would be in simplifying the gizmos implementation for primitives and ensuring that drawing the primitive behaves the same, no matter the isometry.

Solution

Overview

We could abstract the draw functions like gizmos.line(..) on GizmoBuffer to a trait LineBuffer and implement it for GizmoBuffer and wrappers that apply transformations to the lines drawn. We could then use a generic Gizmos: LineBuffer in each implementation of gizmos for primitives. This would also enable us to store gizmos for later use.

Custom GizmoBuffers

We note that all gizmos could be drawn using two functions: gizmos.linestrip_gradient(..) and gizmos.line_gradient(..).
However, we currently incentivise drawing primitives in only one color. As such, two functions like gizmos.line(start: Vec3, end: Vec3) and gizmos.linestrip(positions: impl IntoIterator<Item = Vec3>) would be sufficient to draw every primitive, if we can add the color information later. This would introduce the additional cost that functions like gizmos.ellipse(..) would have to be defined with color on GizmoBuffer and without color on any T: LineBuffer. I don't think the added complexity is justified.

In addition, drawing any primitive using gizmos can be done (and usually that is how they are implemented) by calculating the positions in any line(-strip) of the gizmo as if the primitive was drawn at the origin and then translating and rotating them by applying the specified Isometry.

This motivates creating a trait LineBuffer

trait LineBuffer {
	fn linestrip_gradient<C: Into<Color>>(
		&mut self,
        positions: impl IntoIterator<Item = (Vec3, C)>,
    );
    fn line_gradient<C: Into<Color>>(
		&mut self,
        start: Vec3,
        end: Vec3,
        start_color: C,
        end_color: C
    );
}

which could be implemented by GizmoBuffer and custom structs like IsometryGizmos, a thin wrapper around an underlying GizmoBuffer, that simply applies the provided Isometry to each position drawn.

Any additional gizmo-drawing related function like gizmos.ellipse(..) or gizmos.linestrip_2d(..) present on GizmoBuffer could now be implemented as functions on LineBuffer with a default implementation or via extension traits akin to LineBufferExt that are implemented for all types implementing LineBuffer. This would also make them available to custom GizmoBuffers like IsometryGizmos as well as users of Gizmos in systems.

Implementing .gizmo()

The approach using custom GizmoBuffers would then allow us to define two traits GizmoProvider and GizmoDraw akin to Meshable and MeshBuilder from bevy_mesh.

trait GizmoProvider {
    type GizmoBuilder<'a>: GizmoDraw;

    fn gizmo<'a>(&'a self) -> Self::GizmoBuilder<'a>;

    fn to_gizmo(self) -> Self::GizmoBuilder<'static>;
}
trait GizmoDraw {
    fn draw<Gizmos: LineBuffer>(&self, gizmos: &mut Gizmos, color: Color);
}

GizmoProvider would be implemented for any applicable primitive and could produce a GizmoDraw which could then get a Gizmos type like IsometryGizmos that applies the isometry to all line(-strip)s drawn. Please note that GizmoProvider can be implemented for both 2D and 3D primitives. To distinguish 2D from 3D primitives, a marker trait GizmoDraw2d: GizmoDraw would be neccessary.

By adding an

impl<T: GizmoProvider> GizmoDraw for &T {
    fn draw<Gizmos: LineBuffer>(&self, gizmos: &mut Gizmos, color: Color) {
        self.gizmo().draw(gizmos, color);
    }
}

we can allow drawing primitives without calling .gizmo() on them every time or confusing the compiler. This approach would also guarantee that gizmos.primitive_2d(my_shape.gizmo(), ..) and gizmos.primitive_2d(&my_shape, ..) behave identically.

Having both GizmoProvider::gizmo and GizmoProvider::to_gizmo may seem unneccessary but may be useful for primitives that are not Copy. e.g. drawing a Polygon once should not require cloning the polygon, but always borrowing the vertices of the polygon would prevent storing the builder in a Component and is as such not sufficient for this usecase. GizmoProvider could also be split into GizmoProvider and ToGizmoProvider to reduce boilerplate for Copy primitives.

Drawing primitives

Drawing primitives can now be done by calling gizmos.primitive_Nd(..), which could be implemented on an extension trait for LineBuffer.

pub trait LineBufferPrimitiveExt {
    fn primitive_3d(
        &mut self,
        primitive: impl GizmoDraw,
        isometry: impl Into<Isometry3d>,
        color: impl Into<Color>,
    );

    fn primitive_2d(
        &mut self,
        primitive: impl GizmoDraw2d,
        isometry: impl Into<Isometry2d>,
        color: impl Into<Color>,
    );
}
impl<T: LineBuffer> LineBufferPrimitiveExt for T {..}

Notably, 2D primitives can also be drawn and oriented in 3D space using gizmos.primitive_3d(..).

Using this API would be very similar to the old one. The only difference is that configurations like .resolution(..) would have to be applied on the primitive.

let ellipse = Ellipse::new(4.0, 3.0);
gizmos.primitive_2d(ellipse.gizmo().resolution(128), Isometry2d::IDENTITY, Srgba::RED);
gizmos.primitive_3d(&ellipse, Vec3::Z, Srgba::BLUE);

Components

We can now store a GizmoDraw and Color inside a component. Storing the builder instead of the compiled line(-strip)s allows for easy modification of the configuration of the gizmos. This may be useful for e.g. reducing the resolution of a sphere when the player moves away. By creating a custom struct CompiledGizmo: LineBuffer + GizmoDraw we could also store compiled gizmos if computing the points for a gizmo is very expensive. We would then need to simply transform and copy them over to the actual GizmoBuffer used for drawing.

#[derive(Component)]
#[require(Transform)]
struct GizmoComponent<T: GizmoDraw> {
    pub builder: T,
    pub color: Color
}

Creating such a component requires that the builder is valid for 'static. As mentioned above, this can be achieved by calling .to_gizmo() on the associated primitive. We could also allow modifying the actual primitive itself through GizmoComponent as all T: GizmoProvider are also GizmoDraw. If we were to make e.g. the EllipseBuilder::half_size public, the entire primitive and its configuration could be modified, allowing for a lot of flexibility.

We can then create a system that draws these gizmos with the entity's transform (or at least translation and rotation) applied. Please note that applying scale aswell could be achieved easily by creating something like a struct TransformGizmos: LineBuffer, similar to IsometryGizmos.

Examples

The following is an example implementation of gizmos for Ellipse:

pub struct EllipseBuilder {
    resolution: u32,
    half_size: Vec2,
}
impl EllipseBuilder {
    pub fn resolution(mut self, resolution: u32) -> Self {
        self.resolution = resolution;
        self
    }
}
impl GizmoProvider for Ellipse {
    type GizmoBuilder<'a> = EllipseBuilder;

    fn gizmo<'a>(&'a self) -> Self::GizmoBuilder<'a> {
        self.to_gizmo()
    }

    fn to_gizmo(self) -> Self::GizmoBuilder<'static> {
        EllipseBuilder {
            half_size: self.half_size,
            resolution: 32,
        }
    }
}
impl GizmoDraw for EllipseBuilder {
    fn draw<Gizmos: LineBuffer>(&self, gizmos: &mut Gizmos, color: Color) {
        gizmos.linestrip_2d(
            (0..=self.resolution).map(|i| {
                self.half_size * Vec2::from_angle(TAU * i as f32 / self.resolution as f32)
            }),
            color,
        );
    }
}
impl GizmoDraw2d for EllipseBuilder {}

This is an implementation for a non-Copy type, Polyline3d:

pub struct Polyline3dGizmo<'a> {
    vertices: Cow<'a, [Vec3]>,
}
impl GizmoProvider for Polyline3d {
    type GizmoBuilder<'a> = Polyline3dGizmo<'a>;

    fn gizmo<'a>(&'a self) -> Self::GizmoBuilder<'a> {
        Polyline3dGizmo {
            vertices: Cow::Borrowed(&self.vertices),
        }
    }

    fn to_gizmo(self) -> Self::GizmoBuilder<'static> {
        Polyline3dGizmo {
            vertices: Cow::Owned(self.vertices),
        }
    }
}
impl<'a> GizmoDraw for Polyline3dGizmo<'a> {
    fn draw<Gizmos: LineBuffer>(&self, gizmos: &mut Gizmos, color: Color) {
        gizmos.linestrip(self.vertices.iter().copied(), color);
    }
}

As you can see, both of these implementations are a bit simpler than the previous solution using GizmoPrimitiveNd. In particular, the implementation does not have to concern itself with the concrete type of Gizmos both simplifying it and being more flexible.

With these implementations, we can now do

fn setup(mut commands: Commands) {
	// Spawn an entity with a gizmo-ellipse centered at `Vec3::ONE`. 
	// Moving/rotating this entity would cause the ellipse to move/rotate.
	commands.spawn((
		Transform::from_translation(Vec3::ONE),
		GizmoComponent {
			builder: Ellipse::new(4.0, 3.0).to_gizmo().resolution(128),
			color: Srgba::RED.into()
		}
	));
}

fn draw(mut gizmos: Gizmos) {
	let ellipse = Ellipse::new(1.0, 2.0);
    let polyline = Polyline3d::new([Vec3::X, Vec3::Z, Vec3::Y, Vec3::ZERO]);

	// Draw a 2D shape in 2D, customising its appearance
	gizmos.primitive_2d(ellipse.gizmo().resolution(128), Isometry2d::IDENTITY, Srgba::GREEN);
	// Draw a 2D shape in 3D
	gizmos.primitive_3d(&ellipse, Vec3::Y, Srgba::BLUE);
	// Draw a 3D shape
	gizmos.primitive_3d(&polyline, Quat::from_rotation_z(FRAC_PI_2), Srgba::WHITE);
}

[tigregalis]

I really like the design, particularly how decoupled/deferred it is.

I have done some additional work on my PR which I pushed last night, which I think does actually approach most of the goals outlined here, if not exactly the shape. It doesn't change any of the existing underlying/downstream machinery, only the upstream/high level interfaces.

1. trait ToGizmoBlueprint2d: (effectively GizmoProvider)
   • provides a method that takes a GizmoBlueprint and produces some kind of "blueprint" type
   • the blueprint is the configurable "builder" in the "builder pattern" sense - but I'll call it a "configurer" after this due to the overloading of the term
   • the data for the blueprint/configurer is borrowed from the primitive itself - probably need to take the approach you've outlined with the two methods and the Cow
2. trait GizmoBlueprint: (effectively GizmoDraw)
   • provides a build method that takes as input &mut self and &mut GizmoBuffer
   • this is where the "rendering" work takes place (where users write the commands)
   • this trait is either directly implemented by users on blueprint types, or they can implement the helper trait SimpleGizmoBlueprint2d
   • implementing this trait also
3. trait SimpleGizmoBlueprint2d:
   • users implement a build_2d method directly on a primitive, and they get a ToGizmoBlueprint2d and GizmoBlueprint implementation for free
   • this is useful when you don't really need a configurer and the blueprint provides all the information needed
4. SimpleGizmoBuilder2d:
   • not really user-facing, it's just part of the
   • a blueprint type that simply wraps a primitive
5. trait GizmoPrimitive2d:
   • the extension trait that gives GizmoBuffer the ability to draw primitives
   • this is what gives the Gizmos system parameter and the GizmoAsset asset access to the primitive_2d method.
   • this is automatically implemented for GizmoBuffer for all P where P: Primitive2d + ToGizmoBlueprint2d
   • the output type is a GizmoAssembler<Blueprint>
6. struct GizmoAssembler:
   • a scope guard that contains &mut GizmoBuffer and is produced by GizmoPrimitive2d::primitive_2d
   • provides access to the blueprint configurer via Deref/Mut
   • on drop, it checks if the GizmoBuffer is enabled and runs the blueprint's GizmoBlueprint::build method.
7. impl<B: GizmoBlueprint> From<B> for GizmoAsset:
   • supports gizmo_assets.add(Circle::new(0.5).to_blueprint_2d(Vec2::ZERO, PURPLE).resolution(20))

• Extensibility: It should be possible to implement gizmos for primitives defined outside of bevy_math / bevy_gizmos.

This is supported in the PR by implementing either (impl ToGizmoBlueprint2d for MyPrimitive and impl GizmoBlueprint for MyBlueprint) or impl SimpleGizmoBlueprint2d for MyPrimitive.

• Composability: It should be possible to construct gizmos for two or more primitives at the same time, modify the gizmos and then draw them. This would be particularly useful for primitives that are composed of multiple constituent primitives like Extrusion or Ring.

I've implemented Ring which is generic over any ToGizmoBlueprint2d, with the inner types still configurable via a RingBuilder.

I haven't got dyn DynGizmoBlueprint to work unfortunately, I was hoping to avoid the Cow but it might be necessary. Motivation would be something like implementing GizmoBlueprint for Vec<Box<DynGizmoBlueprint>>.

I had thought about a simple design for a BakedGizmo (you've called it CompiledGizmo):

/// This *could* be collapsed into positions and colors to halve the number of allocations,
/// with a usize field as the delimiter.
///
/// This assumes an IsometryXd::IDENTITY. The actual isometry of the drawn gizmos will be computed from the GlobalTransform.
#[derive(Component, Debug, Clone, Default)]
pub struct BakedGizmo {
    /// Vertex positions for line-list topology.
    pub list_positions: Vec<Vec3>,
    /// Vertex colors for line-list topology.
    pub list_colors: Vec<LinearRgba>,
    /// Vertex positions for line-strip topology.
    pub strip_positions: Vec<Vec3>,
    /// Vertex colors for line-strip topology.
    pub strip_colors: Vec<LinearRgba>,
}

• Component: We could define a GizmoComponent struct and implement Component for it. This component could draw a specified Gizmo after applying the entities Transform to the Gizmo. This could provide a uniform solution for drawing hitboxes or colliders.

The current approach relies on Assets (which are basically precompiled Gizmos) which mirror the meshable primitives system almost precisely, other than the fact that the Gizmo component should be decomposed into smaller components. The GizmoAsset From GizmoBlueprint impl I have provides that. If you wanted to store the primitive or blueprint directly (without unrecoverable type erasure), we'd either need:

1. TypeRegistry: every primitive or blueprint gets registered in the app
2. and/or BSN: I haven't read the proposal in depth, I'm only assuming here, but based on what I've read: essentially, the type itself determines what it wants to fetch from the World and what it wants to insert into the world, so presumably you could store the primitive/blueprint (uncompiled) directly this way

• 2D primitives in 3D: Currently, 2D primitives can be drawn in 2D space using gizmos.primitive_2d(..) and only select 2D primitives like Ellipse or Rect can be drawn in 3D space using functions like gizmos.ellipse(..). We could allow drawing arbitrary 2D primitives in 3D space.

I haven't touched this, but technically this already works for retained gizmos (GizmoAsset) as you can update the transformation, but not for immediate mode gizmos. This does what you'd expect:

fn setup(
    mut commands: Commands,
    mut gizmo_assets: ResMut<Assets<GizmoAsset>>,
) {
    let mut gizmo = GizmoAsset::new();
    gizmo.primitive_2d(&RegularPolygon::new(1.0, 7), Isometry2d::IDENTITY, CRIMSON);

    commands.spawn((
        Gizmo {
            handle: gizmo_assets.add(gizmo),
            line_config: GizmoLineConfig {
                width: 5.,
                ..default()
            },
            ..default()
        },
        Transform::from_xyz(4., 1., 0.),
    ));

    commands.spawn((
        Camera3d::default(),
        Transform::from_xyz(0., 1.5, 6.).looking_at(Vec3::ZERO, Vec3::Y),
        FreeCam::default(),
    ));
}

fn rotate(time: Res<Time>, mut query: Query<&mut Transform, With<Gizmo>>) {
    for mut transform in &mut query {
        transform.rotate_local_y(time.delta_secs());
    }
}

• Common application of Isometry: Drawing a primitive usually involves calculating vertex positions as if the primitive was not rotated/translated and transforming by an isometry later. This process could be unified across all primitives. The advantages would be in simplifying the gizmos implementation for primitives and ensuring that drawing the primitive behaves the same, no matter the isometry.

Strongly related to the previous point I think. It would be nice for Meshable to do the same.

[tigregalis]

(Posting this as a second thread of discussion):

However, after looking at the Meshable and Extrudable traits, seeing that they repeat a lot of logic that's done in Gizmos as well, I'd like to propose something much more radical:

There should be a generic "position generation" trait for primitives; this would probably look very much like Extrudable currently does, except that the positions won't be assumed to be precomputed and then referred to by indices into the vertices but they are "pure" in a sense (I just looked it up, the actual term is analytic), lines are just mathematical descriptions of lines and curves are just mathematical descriptions of curves.

Additionally, realistically, only segments that are curved actually need to be configured by resolution, and that can be done downstream by the consumer of the boundary segment (gizmo line generator, mesh generator, point cloud? sdfs?), using the parameter t. We could also probably support dynamic resolutions based on the viewport. Something like the following sketch:

/// An analytic, resolution-independent definition of a closed 2D shape boundary.
///
/// A `Boundary` defines what is "inside" and "outside" by its winding order.
/// The implementer is responsible for ensuring segments connect continuously.
/// Downstream consumers are responsible for sampling points and generating meshes.
pub trait Boundary {
    /// Returns the analytic segments describing this shape’s boundary.
    fn boundary(&self, isometry: Isometry3d) -> impl Iterator<Item = BoundarySegment>;
}

/// A continuous analytic segment forming part of a boundary.
///
/// Each segment defines its own parameter space (t between 0 and 1).
/// The implementer is responsible for ensuring segments connect continuously.
pub enum BoundarySegment {
    /// A straight line.
    Line(LineBoundary),

    /// A circular or elliptical arc defined by angle sweep.
    Arc(ArcBoundary),
}

/// A line segment starting at `origin`, extending along `direction` for `length`.
/// The normal is right of the tangent
pub struct LineBoundary {
    pub origin: Vec3,
    pub direction: Vec3,
    pub length: f32,
}

/// A circular or elliptical arc centered at `center`, starting at `start_angle` from `Vec3::Y`,
/// sweeping by `sweep_angle` radians.
/// If the sweep is counter-clockwise (positive), the inward normal is "outside" the circle (right of the tangent)
/// If the sweep is clockwise (negative), the inward normal is "inside" the circle (right of the tangent)
pub struct ArcBoundary {
    pub center: Vec3,
    pub radius: Vec3, // allows ellipse when x != y
    pub start_angle: f32,
    pub sweep_angle: f32,
}

pub trait AnalyticSegment {
    /// Evaluate a point along the segment at parameter t, where start is t = 0, and end is t = 1
    fn point_at(&self, t: f32) -> Vec3;

    /// Tangent direction at parameter t, where start is t = 0, and end is t = 1
    fn tangent_at(&self, t: f32) -> Vec3;

    /// Outward-facing normal at parameter t, where start is t = 0, and end is t = 1
    fn normal_at(&self, t: f32) -> Vec3;
}

impl AnalyticSegment for LineBoundary { ... }
impl AnalyticSegment for ArcBoundary { ... }

The above would be able to represent all existing 2d primitives, but it could be extended to Bezier curves for example.
All existing 3d primitives can be represented as an extrusion (circle to cylinder, base to prism) rotation (capsule2d to capsule3d, circle to sphere, circle to torus), loft (circle to cone (point), base to pyramid (point))

[lynn-lumen]

The above would be able to represent all existing 2d primitives, but it could be extended to Bezier curves for example.

I like this approach / idea but to be able to describe every possible shape, you would need something like BoundarySegment::Curve(impl Fn(f32) -> VecN) as some shapes (like an inset ellipse) cannot be accurately represented as polynomials, arcs or lines. It is possible to approximate pretty much every curve as a polynomial, but those polynomials get unwieldy fast and would introduce an additional error term to .resolution.

Meshing a primitive based on its BoundarySegments is also non-trivial and seems as complex as meshing a general non-intersecting polygon. Sure, it is possible, but it seems like it may be slower and lead to worse geometry. I am excited for what the final solution will be like though.