self.parameters of pl.LightningModule

[thistlillo]

Looking at the animation here:

https://www.pytorchlightning.ai/

starting with "1. Computational code goes into LightningModule"

the configure_optimizers(.) method becomes:

def configure_optimizers(self):
    optimizer = torch.optim.Adam(self.parameters, lr=1e-3)
    return optimizer

(see attached image)

However, if I use self.parameters I gen an error:

  File "<REMOVED>/model.py", line 86, in configure_optimizers
    optimizer = torch.optim.Adam(self.parameters, lr=1e-3)
  File "/opt/conda/envs/torch/lib/python3.8/site-packages/torch/optim/adam.py", line 74, in __init__
    super(Adam, self).__init__(params, defaults)
  File "/opt/conda/envs/torch/lib/python3.8/site-packages/torch/optim/optimizer.py", line 47, in __init__
    param_groups = list(params)
TypeError: 'method' object is not iterable

I need to use self.parameters() (with the two parentheses), as in other examples I have found. It makes sense, but...

... the self.parameters (without parantheses) is on the home page, inside of the "official presentation".
Which is the correct one? Why do I get that error?

Regards

[rohitgr7]

it should be self.parameters(). That's a bug on the website.
cc @aribornstein @edenafek

[ananthsub]

@rohitgr7 - ideally it should be even more specific in referencing the encoder/decoder parameters. Otherwise the docmentation can confuse users over whether they should use the LightningModule as an nn.Module or as a system to train modules.

[thistlillo]

@anathsub I am interested in what you wrote since I am using several inner nn modules. Could you please tell me what the correct instruction would be? Should I make a list with the parameters of the inner modules?

[rohitgr7]

what he meant is since you have multiple modules within LightningModule, namely encoder & decoder, while initiating optimizers, we can explicitly set the parameters for the models required to be trained to explicitly tell which modules are being trained for documentation purpose. But since your case contains an auto-encoder structure. self.parameters() would work just fine.

[thistlillo]

Sorry @rohitgr7 , but I did not understand your message.
In my case I do not have an auto-encoder structure. My Lightning module is actually very crowded.

If have several torch.nn.Module modules, managed by a LightningModule, can I keep using only self.parameters() in configure_optimizers() or should I explicitly list the parameters of the torch modules?

[rohitgr7]

Oh sorry, I meant the example in the shared image is an auto encoder. If you want to optimize all the modules in your case then self.parameters() will work just fine.

[dksahuji]

If I have two nn module classes, one for encoder and one for decoder as in GAN.
in the lightning system, I have initialized as

self.encoder = ...
self.decoder = ...

See this pytorch lightning GAN tutorial for reference which does not use self.parameters() .

will encoder and decoder both be iterated with self.parameters() ? Remember in GAN forward pass is just encoder pass.