Read 311990 words (17104 unique) from text file. terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc Aborted (core dumped) #52
Description
/home/dina/code/fastBPE/fast learnbpe 500 /home/dina/CodeGen/data/test_dataset/cpp.sa-cl.tok.shuf.50gb > /home/dina/CodeGen/data/test_dataset/cpp.sa-cl.codes Loading vocabulary from /home/dina/CodeGen/data/test_dataset/cpp.sa-cl.tok.shuf.50gb ... Read 311990 words (17104 unique) from text file. terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc Aborted (core dumped)