Not showing anything if used with react-native-skia

[nmongiya]

I am trying to use react-native-skia with @babylonjs/react-native(1.6.1) and RN 0.71 but can not use both of them together. Only of them is visible on the screen.

I had the same problem earlier with RN 0.70 where skia drawings were not available in debug mode(or when app was connected to metro) but ONLY with reract-native-skia version 0.1.157, I have managed to have both libraries available at the same time.

It prints out the below message but doesn't render anything in EngineView
BJS -: Babylon Native (v5.42.2) launched
I can't think of a reason why this is happening, any thoughts? is it the canvas that is blocked?

[carolhmj]

ping @CedricGuillemet

[CedricGuillemet]

I don't know anything about react-native-skia. TBH, I didn't even know it existed.
It's hard to have a clue on what to check when you don't know how it's architectured.
Do you know anything about that @bghgary @ryantrem ?

[ryantrem]

I also don’t really know anything about it. I expect it would require some debugging. @nmongiya are you testing on Android or iOS? Depending on the platform someone might be able to give some guidance on how to try to diagnose. There might be some debugging tips you can reuse here: https://forum.babylonjs.com/t/im-trying-to-make-babylonreactnative-to-work-on-macos/39178/74

[nmongiya]

I am having this issue on Android only. NO issues on ios.
Couple of interesting things:

1. When Metro is connected : Skia component is available but Babylon is not
2. When Metro is NOT connected: Babylon is visible on screen but skia is not

I thought it could be a race condition to get the canvas and I tried to load Babylon component after couple of seconds timeout and both works fine in that case even when metro is connected, here is what I can see in android logcat

I/SurfaceView: onWindowVisibilityChanged(0) true android.view.SurfaceView{f45f322 V.E...... ......I. 0,0-0,0} of ViewRootImpl@9e07a49[MainActivity]
I/SurfaceView: onWindowVisibilityChanged(0) false android.view.SurfaceView{f2911b3 I.E...... ......I. 0,0-0,0} of ViewRootImpl@9e07a49[MainActivity]
I/SurfaceControl: nativeRelease nativeObject s[526794639040]
I/SurfaceControl: nativeRelease nativeObject e[526794639040]
I/SurfaceControl: assignNativeObject: nativeObject = 0 Surface(name=null)/@0x8c0dc8b / android.view.SurfaceControl.readFromParcel:1117 android.view.IWindowSession$Stub$Proxy.relayout:1820 android.view.ViewRootImpl.relayoutWindow:9005 android.view.ViewRootImpl.performTraversals:3360 android.view.ViewRootImpl.doTraversal:2618 android.view.ViewRootImpl$TraversalRunnable.run:9971 android.view.Choreographer$CallbackRecord.run:1010 android.view.Choreographer.doCallbacks:809 android.view.Choreographer.doFrame:744 android.view.Choreographer$FrameDisplayEventReceiver.run:995
I/SurfaceControl: nativeRelease nativeObject s[526792490368]
I/SurfaceControl: nativeRelease nativeObject e[526792490368]
I/SurfaceControl: nativeRelease nativeObject s[526793921536]
I/SurfaceControl: nativeRelease nativeObject e[526793921536]
I/ViewRootImpl@9e07a49[MainActivity]: Relayout returned: old=(0,0,1920,1200) new=(0,0,1920,1200) req=(1920,1200)0 dur=11 res=0x1 s={true 529446535168} ch=false fn=194
I/SurfaceView: surfaceCreated 1 #8 android.view.SurfaceView{f45f322 V.E...... ......ID 0,0-1917,1002}
I/SurfaceView: surfaceChanged (1917,1002) 1 #8 android.view.SurfaceView{f45f322 V.E...... ......ID 0,0-1917,1002}
I/SurfaceView: applySurfaceTransforms: t = android.view.SurfaceControl$Transaction@b534370 surfaceControl = Surface(name=SurfaceView - com.emesent/com.emesent.MainActivity@f45f322@0)/@0x9a006e9 frame = 194
I/ReactNativeJS: BJS - [13:49:01]: Babylon Native (v5.42.2) launched

[CedricGuillemet]

when both work fine, do you mean both can draw to the canvas?
when Skia is available but babylon is not, does it mean babylon still renders but result is not visible or does it mean the render loop is not started?

[nmongiya]

1. Yes, When I said "Both work fine", I meant that both can draw to the canvas.
2. When Skia is available but babylon is not, It meant, babylon still renders but result is not visible(It's just a black screen)
3. Another interesting find is : When I change the androidView='TextureView' in EngineView, both Skia and Babylon works fine in debug mode (even When Metro is connected) but if SKIA gets drawing first, you can not see anything in babylon. :(

[wcandillon]

@CedricGuillemet we also didn't know you were doing such cool things as well :) Let's discuss this offline.

[CedricGuillemet]

3. Another interesting find is : When I change the androidView='TextureView' in EngineView, both Skia and Babylon works fine in debug mode (even When Metro is connected) but if SKIA gets drawing first, you can not see anything in babylon. :(

This is interesting. It looks like a view stacking problem. I bet that running on a different phone will give a different ordering problem. My guess is some views are opaque, some are not. depending on the order, you see something or not.
It's possible for the Babylon View to be transparent depending on alpha value. is there the same mecanism with Skia? can you try to force transparency on both views?

[nmongiya]

I tried setting isTransparent={ false } in EngineView but that doesn't help.

[CedricGuillemet]

I would try to have everything transparent: Skia and Babylon

[nmongiya]

Setting both transparent doesn't help either.

I do not how babylon works under the hood for react-native, does it use Canvas as well like Skia and can it be locking issue as they mentioned here
https://github.com/Shopify/react-native-skia/blob/main/docs/docs/canvas/contexts.md

[CedricGuillemet]

On Android, depending on transparency, a SurfaceView or a TextureView is created:

BabylonReactNative/Modules/@babylonjs/react-native-iosandroid/android/src/main/java/com/babylonreactnative/EngineView.java

Line 99 in aafbad8

if (androidView.equals("TextureView")) {

Same mecanism as iOS where a MTKView inheritance is created and transparency is enabled by a flag.

I guess Skia creates a view as well. I believe it's possible to get view hierarchy with AndroidStudio and compare when rendering correctly or not.

[nmongiya]

View hierarchy with AndroidStudio's Layout inspector is not very helpful. It doesn't show view attributes that we are looking for.

[chrfalch]

Hi! RNSkia always uses a TextureView when rendering - could the issue here be that combining TextureViews (Skia) and SurfaceViews (Babylon) causes this behaviour?

Did anyone see a difference when setting transparency on the Babylon View to force it to render a TextureView?