Unable to Reproduce Reported Metrics – Request for Frame Extraction Code

[MillerAnacleto]

Hi, I followed the provided steps to test your code, but I wasn't able to achieve the metrics reported in the paper. I suspect the discrepancy might be due to differences in how the frames were extracted from the HMDB and UCF datasets.

Would you be able to share the exact code or methodology used for frame extraction? This would help ensure reproducibility.

Thanks in advance!

[KunyuLin]

Thank you for your attention. I extracted the frame based on ffmpeg, and I think there are some public sources providing frames of UCF.

However, according to my experience, frame extraction would usually have minor influence. I guess there could be some other issues. What performance you have reproduced?

[MillerAnacleto]

On the three tests:

• hmdbucf

1. Best Prec@1: 92.896175 and Corresponding Test Prec@1: 77.103036 (in epoch 24)
2. Best Prec@1: 93.169399 and Corresponding Test Prec@1: 76.903932 (in epoch 14)
3. Best Prec@1: 93.715847 and Corresponding Test Prec@1: 76.057740 (in epoch 17)

• ucfhmdb

1. Best Prec@1: 99.753695 and Corresponding Test Prec@1: 52.944444 (in epoch 15)
2. Best Prec@1: 100.000000 and Corresponding Test Prec@1: 57.722222 (in epoch 18)
3. Best Prec@1: 100.000000 and Corresponding Test Prec@1: 58.055556 (in epoch 16)

I will try extract them on ffmpeg.

[KunyuLin]

OK. I think the reproduced results here fall within the error range of the results reported in our paper (except the 1st result on UCF-to-HMDB). I think the performance gap mainly comes from some data or environment differences. To obtain higher performance, I think you can check the these differences or adaptively modify some hyperparameters.