understanding of overfit_batches

[talhaanwarch]

I am but confused about overfit_batches, if in case of overfit_batches, if the model performance is less than 100%, then it mean that we have some issue, because we are training and validating on same batches. is it correct?
In my case the performance is around 80%, i am not sure in case of overfit_batches, is it acceptable or there is issue
Do we need to track training metric, val metric or both in case of overfit_batches?

[rahulvigneswaran]

I am but confused about overfit_batches, if in case of overfit_batches, if the model performance is less than 100%, then it mean that we have some issue, because we are training and validating on same batches. is it correct?
This is correct.

There might be two reasons for your 80% accuracy:

• Didn't train for enough epochs. Try increasing the epochs or set max_epochs to -1. Also, try reducing the overfit_batches size.
• If it is still not 100%, then there is something wrong with your model/loss/code.

Also, I would suggest you look at the training loss rather than training accuracy.

[talhaanwarch]

by increasing epochs, my training accuracy reaches to 100% but validation accuracy is not crossing 50%? any idea

[rahulvigneswaran]

@talhaanwarch Can you confirm that the training and validation sets are the same in this case?

[talhaanwarch]

thanks reducing the size of overfit_batches help to resolve the issue

[mariovas3]

I am currently having the same "issues". I tried some manual hacking - setting the validation batch be the same as the training batch and then I fit everything perfectly. However, I don't think overfit_batches=1 sets the validation batch to be the same as the training batch and this way I only overfit the actual training batch and my validation batch performance is not great (expected since the batches are different).

Technically the docs are not wrong that overfit_batches just limits the data and disables shuffling in dataloaders, but in the case when you log more stuff in the validaiton_step (the things you care about, other than loss) it would be helpful to see that they improve as well for the batch(es) you are training on.