Prevents joint meshes from appearing before hand tracking is active

[simonedevit]

This PR fixes a visual glitch where a ghost icosphere (joint meshes) appears at the scene origin when entering an immersive session using only controllers (no hand tracking).

Joint meshes are created as soon as WebXRHandTracking feature is attached. If hand tracking is not active yet, those meshes can remain visible until a hand is created and updateFromXRFrame applies the correct visibility. This fix makes the joint meshes invisible immediately on creation, preventing them from appear at the scene origin before hand tracking becomes active.

@matthargett, do you see any issues with this approach?

Additionally, the original mesh visibility has been restored.

Alternative

If we decide not to adopt this fix, users can avoid the ghost joint meshes by explicitly setting:

jointMeshes: { 
   invisible: true
}

during the creation of the default experience (invisible is set to false by default).

---

Forum: https://forum.babylonjs.com/t/request-for-minimal-working-vr-controller-v2-physics/62103/5

[matthargett]

My main problem is that it side-steps the root issue. Why consume these resources at all when there's no hand data, and it may never arrive? isVisible = false hides the visual glitch but doesn't address the real costs:

• several dozen mesh instances + source mesh are created eagerly — GPU buffer uploads during the critical first frames of XR session entry, creating jank on lower-end (anything except AVP) devices.
• Scene graph pollution — those dozens of nodes evaluated every frame by the engine even when invisible (bounding box checks, transform updates)
• Physics overhead — if enablePhysics is true, dozens of physics shapes are created for hands that may never get data from input sources
• Two GLB loads + parse kicked off immediately even if the user only has controllers

I think the right fix is to defer _GenerateTrackedJointMeshes and _GenerateDefaultHandMeshesAsync until the first onControllerAddedObservable fires with an inputSource that actually has a .hand property. I took a quick look at pmndrs/xr, and this is what they do. Godot XR isn't quite as efficient.

I hope this helps! :D

[Popov72]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[bjsplat]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[bjsplat]

Snapshot stored with reference name:
refs/pull/17905/merge

Test environment:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/17905/merge/index.html

To test a playground add it to the URL, for example:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/17905/merge/index.html#WGZLGJ#4600

Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves):

https://playground.babylonjs.com/?snapshot=refs/pull/17905/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/17905/merge
https://gui.babylonjs.com/?snapshot=refs/pull/17905/merge
https://nme.babylonjs.com/?snapshot=refs/pull/17905/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/17905/merge#BCU1XR#0

If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools.

[bjsplat]

Interaction tests

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/17905/merge/testResults/interactionplaywright/index.html

[simonedevit]

Thanks for the insight. You've raised some very valid points regarding the overhead of eager instantiation and scene graph pollution.

My primary goal with this PR is strictly to provide a surgical fix for the visual "ghosting" regression at the origin, which is currently impacting the user experience.

Regarding the root issues you mentioned (and as you suggested), I propose a dedicated follow-up PR (after aligning with the core team) to implement a proper deferred initialization. We could:

• Prefer setEnabled over isVisible for inactive joint meshes to reduce per-frame work until we have a valid hand pose.

• Deferred generation: keep the GLB assets preloading (to ensure hands can be displayed instantly), but defer calling of _GenerateTrackedJointMeshes and invoking the _GenerateDefaultHandMeshesAsync callback until hand data is actually present.

[bjsplat]

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/17905/merge/testResults/webgpuplaywright/index.html

[bjsplat]

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/17905/merge/testResults/webgl2playwright/index.html

[matthargett]

I'll leave it up to the BJS maintainers if they want to have this amount of shifting across releases. It does seem like you could patch your own app to disable/hide the joints in the meantime, right?

If you'd like me to dive in with the proper fix, I'm happy to do it :)

[deltakosh]

I tend to agre with @matthargett here. I will let @RaananW decide ultimately.

As we are in code freeze for babylon v9, I'll switch that PR to draft. Raanan should be able to comment after the release

Thanks a ton @simonedevit !

[simonedevit]

Glad to help. If you'd like, I can open a pull request in the documentation repo to track of the following breaking changes.

Starting with version 8.46.1:

• Users must set jointMeshes: { invisible: true } to avoid seeing a ghost sphere in immersive mode when hand tracking is enabled but not in use.
• keepOriginalVisible property no longer has any effect.

[RaananW]

Hi everyone, sorry for a delayed response.

It is true that consuming these resources is not needed, especially if there is no hand data available. Attach should actually be filtered if hands are not supported, but it does execute if hands are supported, even if they are not available. But here comes the interesting question - should a user switching from controller to hand expect it to happen instantly, or is it ok to have a few seconds wait time.
Having the resources in the scene graph might not be the best, but having them ready is what allows controller-hand switches to happen in real time.
Again - I agree that in a lot of the cases having the hands is not needed. so we need to find a common ground here. I assume one way would be to provide the user with a configuration flag to decide if the meshes load on session start on on hand found. It is not a hard task, but it will have to wait till 9.0.0 is released, as we are in code freeze.

Just to address the main issue this PR solves - I would see that as a bug - And your solution is perfectly acceptable. But anyhow, we will wait with it for now.