Shader pipeline log for automatic compilation in loading screens

[JMS55]

What problem does this solve or what need does it fill?

Pipelines take a while to compile. Ideally we compile all that ahead of time in a loading screen, rather than either stutter or pop in during gameplay.

What solution would you like?

Some kind of log to record created pipelines during runtime, and save it to a file. Then, on startup, read the file and queue all the pipelines.

We'll need to handle things like TAA that don't use the Material API, and therefore unconditionally create the pipeline without caching it and deduplicating with existing pipelines in the pipeline cache (which doesn't deduplicate anything in its own).

[alice-i-cecile]

@JMS55 does the edited title make sense? Trying to make sure this is easy to search :)

[JMS55]

I'm on mobile quickly jotting down my thoughts, but yeah mostly makes sense. I would remove the on disk caching, we can't (more complicated issue) cache the pipelines themselves on disk, but we can keep a log of the pipeline descriptors, and then queue their creation on startup every app run.

[mattzque]

This would be very welcome, I couldn't find an easy way to accomplish this.

Some kind of log to record created pipelines during runtime, and save it to a file. Then, on startup, read the file and queue all the pipelines.

would this work for a game with multiple maps each with different materials? Wouldn't we want an ergonomic way to programmatically preload a pipeline and check its state?

[JMS55]

The way I envision this working is via events. You'd send a ShaderPipelineLogEvent::StartRecording { asset_path: String } event, and then later ShaderPipelineLogEvent::StopRecording. Then there would be a PipelineCache method to queue pipelines from a log of descriptors.

Then we could make a default plugin/cargo "dev pipeline log" feature that does a simple setup where it reads an existing log on startup, sends the StartRecording event, and then on app close sends StopRecording and replaces the old log with the new one.

More complicated games with multiple "maps/levels" would have to manually manage the logs themselves, and clearing the cache and loading new logs on level change. You'd probably also want to hide it behind a loading screen, same as loading your mesh and material assets.