Skip to content

Fix handling of new Satpy writer return values#237

Merged
mraspaud merged 2 commits intopytroll:mainfrom
djhoese:bugfix-writer-returns
Sep 11, 2025
Merged

Fix handling of new Satpy writer return values#237
mraspaud merged 2 commits intopytroll:mainfrom
djhoese:bugfix-writer-returns

Conversation

@djhoese
Copy link
Copy Markdown
Member

@djhoese djhoese commented Sep 11, 2025

This fixes #236 by only handling new Satpy return types. It's possible it works with old Satpy since trollflow2 is now directly using split_results from Satpy which should do the heavy lifting of "what did the writer return?".

I'm tempted to also adapt this to now use Delayed at all for the most performance, but I think this should have a discussion. For example, some of the writers (geotiff, simple_image, and even AWIPS tiled) will return a dask array where each chunk (usually 1) of the array is a Path or str of the filename that was saved on disk. Taking advantage of this could probably improve trollflow2 handling of generated files as they are created. For example, pass the filename Array object to map_blocks where the map_blocks function is the trigger for the rest of processing (moving the file to final storage).

The merge of this should be coordinated with #233.

@djhoese djhoese added the bug Something isn't working label Sep 11, 2025
@djhoese
Copy link
Copy Markdown
Member Author

djhoese commented Sep 11, 2025

CI failing still. Ok so apparently I need to test this with stable Satpy and make it work with old and new.

@djhoese
Copy link
Copy Markdown
Member Author

djhoese commented Sep 11, 2025

Ok so this works with old Satpy and old trollimage. This also works with new Satpy (main) and new trollimage (1.27.0, released yesterday), BUT it does not work with old Satpy and new trollimage.

Oh, maybe I can handle this...

@codecov
Copy link
Copy Markdown

codecov Bot commented Sep 11, 2025

Codecov Report

❌ Patch coverage is 95.83333% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 94.95%. Comparing base (d69a75e) to head (76683dc).
⚠️ Report is 40 commits behind head on main.

Files with missing lines Patch % Lines
trollflow2/plugins/__init__.py 95.83% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #237      +/-   ##
==========================================
- Coverage   95.92%   94.95%   -0.97%     
==========================================
  Files          14       14              
  Lines        3311     3309       -2     
==========================================
- Hits         3176     3142      -34     
- Misses        135      167      +32     
Flag Coverage Δ
unittests 94.95% <95.83%> (-0.97%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Member

@mraspaud mraspaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks a lot for the quick response!

@mraspaud mraspaud merged commit d3c7469 into pytroll:main Sep 11, 2025
7 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

callbacks fail with latest trollimage or satpy

2 participants