Conversation
|
Not the most efficient way (the unigrams will have useless trie pointers and backoff). Should really be its own implementation but Holger Schwenk wants something too. . . so sure. |
|
Yes, I understand that a separate implementation is more appropriate in terms of efficiency. In building a classifier, I need to use from unigram to pentagram as features. It will be wonderful to use KenLM as a unified solution to manage these features. So I've made the code change and submit this pull request for your consideration. |
|
Hey @changq , How do you create a lm.binary after you create your unigram ? |
change it to support ngram-order=1 (tested for TrieModel)