LightningCLI: callbacks based on model

[adamjstewart]

I'm trying to use LightningCLI to replace TorchGeo's train.py script. I'm absolutely loving it so far, but have encountered one thing I'm not sure how to translate. The performance metric we monitor in our ModelCheckpoint/EarlyStopping callbacks depends on which model we select (e.g., classification: 'val_loss', object detection: 'val_map', SSL: 'train_loss'). What would be the most idiomatic way to handle this?

My first thought was to override LightningCLI.instantiate_trainer so that we can add callbacks based on parsed model config. Is there a cleaner way to do this? Ideally it would be possible for the user to override this but it's not a requirement.

[tshu-w]

Maybe you can config callbacks for model directly through LightningModule.configure_callbacks

[adamjstewart]

I don't know how I've never discovered this before but this makes so much more sense, thanks!