馃殌 Feature
Unlike the ModelCheckpoint callback EarlyStopping has the min_delta argument, which controls the criteria for training stopping, I.e how much the monitored value must be improved to continue further training. Such behavior provides you with minimal regularization to your model and saves some time because you wouldn't continue training if your model improves a little bit, e.g +0.0001, which sign of overfitting to the validation dataset.
However, when using ModelCheckpoint with or without EarlyStopping I can't control this behavior and maybe my best checkpoint will be rewritten with the overfitted model.
This problem will be really strange in research.
Motivation
The motivation of this proposal is to reduce the chance of overfitting the validation dataset. I didn't meet similar issues like this one before.
Pitch
I suppose to add a new argument to the ModelCheckpoint/for controlling the saving checkpoint relatively on how much the monitored value was improved.
Alternatives
Unfortunately, I don't see any alternatives.
Outline
Also, I would to join one team to write the code of ModelCheckpoint for the min_delta argument, of course, if the issue will be agreed upon.
cc @Borda @carmocca @awaelchli @ninginthecloud @jjenniferdai @rohitgr7
馃殌 Feature
Unlike the
ModelCheckpointcallbackEarlyStoppinghas themin_deltaargument, which controls the criteria for training stopping, I.e how much the monitored value must be improved to continue further training. Such behavior provides you with minimal regularization to your model and saves some time because you wouldn't continue training if your model improves a little bit, e.g +0.0001, which sign of overfitting to the validation dataset.However, when using
ModelCheckpointwith or withoutEarlyStoppingI can't control this behavior and maybe my best checkpoint will be rewritten with the overfitted model.This problem will be really strange in research.
Motivation
The motivation of this proposal is to reduce the chance of overfitting the validation dataset. I didn't meet similar issues like this one before.
Pitch
I suppose to add a new argument to the
ModelCheckpoint/for controlling the saving checkpoint relatively on how much the monitored value was improved.Alternatives
Unfortunately, I don't see any alternatives.
Outline
Also, I would to join one team to write the code of
ModelCheckpointfor themin_deltaargument, of course, if the issue will be agreed upon.cc @Borda @carmocca @awaelchli @ninginthecloud @jjenniferdai @rohitgr7