Performance drops when setting persistent_workers=True

[siantonelli]

I encounter a performance drop when setting persistent_workers equals True in the datamodule. Should it just handle the workers? Why does it affect in some way also the optimization of the model?

A working Colab to reproduce the behavior.