Multiple Sequential trainings slows down speed

[HadiSDev]

Hi.
I have a task where I need to run a training script multiple time with a for-loop like this:

for dataset in datasets:
      ... Training script with datamodule, model, Trainer Init and trainer.fit()

However, after each loop the the training it self slows down incrementally (epoch / sec). I am thinking it is because I need to reset something but I have not been able to find that information. I am using Torch-cpu 2.0.0

Any idea on what I am doing wrong? :(

[HadiSDev]

I tried to remove the neptune logger and model saving functionality and it seemed to have solved the issue