When using use_tmp_file or staging_dir, move/rename files as soon as they are finished

[gerritholl]

Feature Request

Is your feature request related to a problem? Please describe.

I'm using use_tmp_file or staging_dir to prevent files being shipped out by our file distribution system prematurely. Currently, trollflow2 stores all files to temporary files, then renames/moves them when they are all finished. It would be desirable to move/rename each file as soon as it's finished, so that products are delivered to users sooner.

Describe the solution you'd like

I would like that (optionally?) files are renamed to their final name as soon as they're finished.

Describe any changes to existing user workflow

None if the behaviour is optional. But I think I'm the only user anyway.

Additional context

Make the production so fast that it doesn't matter ;-)

[gerritholl]

This may be easier said than done. The save_datasets plugins first collects all the dask graphs, then computes them all simultaneously. Maybe moving the file could be added as a task to the dask graph in some way?

[gerritholl]

May this also be relevant for the FilePublisher? I think that when using FilePublisher, it also only published when all products have been produced, losing the opportunity to ship out files as soon as they are finished?

[gerritholl]

Maybe #138 can be of help here?