Is it possible to use the ExternalTexture plugin within BabylonReactNative?

[tomyates]

There's a plugin for BabylonNative that allows you to use external textures:
https://github.com/BabylonJS/BabylonNative/tree/master/Plugins/ExternalTexture

Is it possible to rebuild BabylonReactNative to include this plugin and can anyone point me in the direction of how I might do this?

Thanks!

[bghgary]

Not currently. At a minimum, we will need to add this plugin to the cmake dependency list. I'm not sure how it would then be exposed to the user since it's a C++ contract. @ryantrem any thoughts on this?

[tomyates]

Thanks @bghgary! Any more info would be great, otherwise we'll have to go down the route of migrating to swift.

[bghgary]

I'm not sure how migrating to swift will help?

[tomyates]

@bghgary sorry, I wasn't clear. I meant as a last resort I'd migrate the whole project to Swift and I could use Babylon Native with the plugin.

[ryantrem]

We do have one (experimental) native API exposed from Babylon React Native currently, which you can see here (specifically XrAnchorHelper.h and XrContextHelper.h): https://github.com/BabylonJS/BabylonReactNative/tree/master/Modules/%40babylonjs/react-native/shared
These were created to expose the platform specific underlying XR anchors so we could augment Babylon React Native with separate react native packages that do things with anchors (like syncing with cloud anchors). I suspect we could do something very similar for ExternalTexture, which I think would involve:

• Adding the plugin to the dependency list as @bghgary mentioned.
• Having a wrapper for ExternalTexture with a new header that is included in the Babylon React Native package, but with some modifications compared to ExternalTexture.h:
  • Have AddToContextAsync take a jsi::Runtime instead of a Napi::Env since jsi::Runtime would be more natural in the context of a React Native app, and the wrapper could construct a Napi::Env from the jsi::Runtime.
  • Either also include <Babylon/Graphics/RendererType.h> which has the definition of TextureT, or do something less nice like just exposing a uint_32t pointer that has to be cast (which is basically what was done for the anchor thing).

I think if we did those things, then a consuming app could use those headers in its own native c++ code inject external textures. I also see that ExternalTexture.h includes <Babylon/JsRuntime.h> though I didn't see any usage of it, so I don't think this is an actual dependency we'd need to worry about, but @bghgary can confirm.

Also, I don't think ExternalTexture is implemented for some platforms (Android?), so that may also be a blocker for you.

[tomyates]

@ryantrem thank you Ryan for your detailed explanation and thoughts. I'll think about this over the weekend and reply next week.

[tomyates]

@ryantrem @bghgary Thanks again - not having ExternalTexture implemented in Android would not be a blocker for us.

This approach looks really promising and is something we'd love to get working. Is this something you could implement? We'd be happy to help in anyway we can.
Thanks!

[tomyates]

Just as a sanity check - If we had Babylon Native setup and configured in an app, is it possible to wrap this as React Native module? Or is this ridiculous and not possible?

[ryantrem]

This approach looks really promising and is something we'd love to get working. Is this something you could implement? We'd be happy to help in anyway we can. Thanks!

I will try to chat with @bghgary some about the best way to expose some BN types/headers with BRN. Once we have an idea of the right way to approach that, then I think something could be implemented, and I expect we'd want some help from you at this point. :)

Just as a sanity check - If we had Babylon Native setup and configured in an app, is it possible to wrap this as React Native module? Or is this ridiculous and not possible?

If you mean you have Babylon Native setup and working in isolation, and then bring that into a React Native app as your own custom module, I would say yes you can do that, but: 1) You would end up with two different JS engines (or at least contexts) and 2) The JavaScript you write against the Babylon.js API with Babylon Native would be isolated from the JavaScript in the consuming React Native app. What Babylon React Native largely does is connect Babylon Native to the JavaScript engine instance being used by the consuming React Native app.

[bghgary]

@tomyates Sorry for the delay, but @ryantrem and I finally discussed this yesterday. It should be possible to expose this, but we currently don't have an existing pattern for doing it. The ExternalTexture from Babylon Native needs to be wrapped inside an asynchronous function exposed through some public contract from Babylon React Native. If this sounds like something you would be interested to contribute, we can discuss more details on the contract shape.