Lower accuracy on Kinetics than reported in the paper

[shanshuo]

Thanks a lot for open-sourcing the code. I follow the instruction of SSv2 to prepare the Kinetics. Both the validation set and test set category ID starts from 0, as mentioned in the issue. On Kinetics, I follow the same parameters used in SSv2 to train:

python -u main.py kinetics RGB --arch resnet50 --num_segments 8 --lr 0.0001 --lr_steps 20 --epochs 25 --batch-size 32 --workers 2 --dropout 0.5 --root_log checkpoints/kinetics_weights/ --root_model checkpoints/kinetics_weights/ --wd 0.0005 --gpus 0 --episodes 600

I train on 1 A100 GPU with PyTorch 1.10.0. Here is the training log.csv.
I used the checkpoint best performed on the validation set ckpt24.best.pth.tar for testing. The 1-shot mean accuracy is 72.47, 1.79 lower than reported 74.26. But on SSv2 I obtain 44.30, higher than 43.82 reported in the paper. I'm not sure about the reason.
Could you please share the parameters for training the Kinetics?

[khoiucd]

Hi Shou Chen,

Thank you very much for your interest in our work!

The implementation is slightly different from what was reported in the paper. For example, I have added an entropy loss term in training:

fsvc-ata/train.py

Line 39 in bf1cb7d

ent = -(norm_D.mean(1) * torch.log(norm_D.mean(1) + 1e-12)).sum(1)

These updates boost the performance of SSv2, higher than what was reported in the paper.

I suggest you vary the learning rate, the learning rate schedule, and the number of training epochs when training on the Kinetics dataset.

At the moment, I cannot provide the parameters for the Kinetics dataset for you. I will examine the code and send them to you as soon as possible.