Proper Iteration/Experimentation using LightningModule

[norbertmarko]

Hello!

I have a question regarding the proper usage of LightningModules. The StyleGuide says that the LitModule should be self contained. This includes the model parts, the loss, metrics etc.

Now, I understand that if we change something about the model, we can treat it as a different experiment and create a new module for it.
Where I'm a little bit fuzzy is the switching of loss functions, metrics and such between experiments. Since these are usually listed as the part of the module, should I create a new module class for every iteration (e.g. for a research)?

How should I approach this problem?

Thanks in advance!

[mauvilsa]

In the style guide you can see the recommendation about being specific in the module's init. There you can see one parameter defined as encoder: nn.Module. This essentially makes a submodule (encoder) configurable. No need to implement multiple LightningModule's to switch between different encoders. The same concept can be applied for loss, metrics, optimizers, schedulers, etc.

If you want to make things much more configurable for research, you might want to look at CLIs. Read the short introduction for the motivation why. Using that, you can see a similar pattern than in the style guide but for making optimizers and schedulers configurable, see multiple-optimizers-and-schedulers.