speed of training

[jlkazdan]

Hello!

Thanks for answering my earlier questions. I also wanted to ask you how long your DPO training takes. I am running with the stage3 deepspeed config, and it is taking roughly 8 hours on 8xa100 to run DPO on your dataset. Is this what you observed too? It seems very slow to me.

Thanks for the help!

Josh

[jlkazdan]

And one other question-- I am measuring much lower accuracies than the ones that you reported in the paper for the SFT'ed models. Specifically, after SFT I'm measuring:

resultsAcc-arc_easy_test=83.333-arc_challenge_test=72.526-openbookqa_test=83.400-siqa_test=61.003-gsm8k_test=44.200

[Raibows]

Hi,

training speed

I think DeepSpeed Zero Stage 3 will significantly degrade the training speed. Could you please try Stage 2 config?

SFT performance

Is the training loss normally descending? If so, I think it is the overfitting problem. I suggest trying SFT with 1 or 3 epochs.

[zzwjames]

And one other question-- I am measuring much lower accuracies than the ones that you reported in the paper for the SFT'ed models. Specifically, after SFT I'm measuring:

resultsAcc-arc_easy_test=83.333-arc_challenge_test=72.526-openbookqa_test=83.400-siqa_test=61.003-gsm8k_test=44.200

I also cannot reproduce the results.