Specifying a shader pipeline within Kage code

[bandaloo]

Hi, I'm very interested in how ebiten deals with fragment shaders, and how you implemented a go-like DSL for cross-compiling to different shading languages. I happened to create a similar thing, called tinsl, which is a sort of extension to GLSL with type inference using :=. Beyond this, I was also interested in being able to render to texture half-way through a shader, and read from that texture later in the shader, essentially specifying an entire rendering pipeline in shader-like code. The tinsl compiler will split this up into multiple shaders. Consider this example of doing a "bloom" effect:

def threshold 0.5 // threshold for luma to mask off

fn luma(vec4 color) { return dot(color.rgb, vec3(0.299, 0.587, 0.114)); }

// linear gaussian blur; we do a horizontal then vertical blur in two separate shader
// passes to get a radial blur, which is much more efficient than a single pass
fn blur(vec2 direction, int channel) {
  uv := npos;
  off1 := vec2(1.3333333333333333) * direction;
  mut color := vec4(0.);
  color += frag(uv, channel) * 0.29411764705882354;
  color += frag(uv + (off1 / res), channel) * 0.35294117647058826;
  color += frag(uv - (off1 / res), channel) * 0.35294117647058826;
  return color;
}

// reusable "procedure" for doing a horizontal blur, then vertical, then repeating it
pr two_pass_blur(float size, int reps, int channel = -1) {
  loop reps {
    blur(vec2(size, 0.), channel); refresh;
    blur(vec2(0., size), channel);
  }
}

// mask off the bright fragments, then render out to off-screen texture 1
{ vec4(frag.rgb * (step(1. - luma(frag0), 1. - threshold)), frag.a); } -> 1

// read from texture 1, blur it three times then render back to texture 1
1 -> { @two_pass_blur(size: 1., reps: 3);} -> 1

// add texture 0 (the original image) and texture 1 (the masked blurred image)
// and implicitly render to screen
{ frag0 + frag1; }

This gets compiled into four separate GLSL fragment shaders (a sort of "shader tree") which then get run in a specific sequence. I think such a feature would work really well within ebiten, since you already have a shader DSL. This would enable the user to do much more complex post-processing effects that cannot be accomplished efficiently within a single shader. (Like Kage, tinsl also doesn't have structs yet, because parsing is a lot of work lol)

[Zyko0]

For the moment I think you can handle the N shader invocations in the order you want, in a user function responsible for the post-processing effects, right ?
I'm not sure, I understood the benefits of your suggestion, is it for better shader management ? Like they would be bound already, instead of having to give them specific folder and a specific go-function to handle the multiple calls in order ?

[hajimehoshi]

Thank you for posting!

So, your idea is basically enabling a complex pipeline involving multiple offscreen textures in one shader program, right? This seems very interesting, but probably Ebiten won't adopt this as this would not match with the current API model and implementation so far.

I appreciate you posted your idea in this discussion anyway. Opinions or suggestions from various perspectives are very insightful.

[bandaloo]

Yes, that's exactly right! I figured such a thing might not be a perfect fit for Ebiten but it's fun to think about. Thanks for taking the time to read my thoughts about it.

[hajimehoshi]

BTW,

1 -> { @two_pass_blur(size: 1., reps: 3);} -> 1

This seems the rendering source and target are the same. Does this really work, or does @two_pass_blur uses an offscreen inside?

[bandaloo]

Good question! Yeah, it's not really the same texture since you can't read from and write to the same texture, like you pointed out. In my implementation it renders to a separate texture (call it "tex back") and then tex back becomes the new texture 1. Textures get swapped around like this and it works because they are all the same size. But the user doesn't have to know that, and they can pretend like they are reading to and writing from the same texture. I made an online playground. (Beware there are probably some bugs) https://bandaloo.fun/playground

[hajimehoshi]

OK, thank you for elaborating!