There are discrepancies between the parameters in the paper and those in main.py.

[LiJiaBei-7]

I noticed in the supplementary material that the number of steps is 50,000, but in main.py, steps_per_epoch=500. I would like to ask if this is a mistake? Additionally, the batch_size and gradient accumulation are also different from what was used in the paper.

[Vladimir2506]

@LiJiaBei-7 Thank you so much for pointing this issue out, please set the hyper-parameters following the paper. I modified the default parameters when testing the maximum batch size on 80GB GPUs. I will update a new version of them along with the debug of your next issue on poor training results.