Aggregating gradients before optimizer step

[spivakoa]

Hello,

I would like to aggregate model gradients several times before the optimizer step.

In pytorch I would do something like this:

for _ in range(4):
pred = model.forward()
loss = Loss_function(pred, gt)
loss.backward()
optimizer.step()

How is it possible to achieve this in lightning?

Thank you,
Alex.