Fix handling of new Satpy writer return values#237
Conversation
|
CI failing still. Ok so apparently I need to test this with stable Satpy and make it work with old and new. |
|
Ok so this works with old Satpy and old trollimage. This also works with new Satpy (main) and new trollimage (1.27.0, released yesterday), BUT it does not work with old Satpy and new trollimage. Oh, maybe I can handle this... |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #237 +/- ##
==========================================
- Coverage 95.92% 94.95% -0.97%
==========================================
Files 14 14
Lines 3311 3309 -2
==========================================
- Hits 3176 3142 -34
- Misses 135 167 +32
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
mraspaud
left a comment
There was a problem hiding this comment.
Looks great, thanks a lot for the quick response!
This fixes #236 by only handling new Satpy return types. It's possible it works with old Satpy since trollflow2 is now directly using
split_resultsfrom Satpy which should do the heavy lifting of "what did the writer return?".I'm tempted to also adapt this to now use Delayed at all for the most performance, but I think this should have a discussion. For example, some of the writers (geotiff, simple_image, and even AWIPS tiled) will return a dask array where each chunk (usually 1) of the array is a Path or str of the filename that was saved on disk. Taking advantage of this could probably improve trollflow2 handling of generated files as they are created. For example, pass the filename Array object to
map_blockswhere themap_blocksfunction is the trigger for the rest of processing (moving the file to final storage).The merge of this should be coordinated with #233.
flake8 trollflow2AUTHORS.mdif not there already