Package edu.berkeley.nlp.lm.io
Class GoogleLmReader<W>
java.lang.Object
edu.berkeley.nlp.lm.io.GoogleLmReader<W>
- All Implemented Interfaces:
LmReader<LongRef,
NgramOrderedLmReaderCallback<LongRef>>
public class GoogleLmReader<W>
extends Object
implements LmReader<LongRef,NgramOrderedLmReaderCallback<LongRef>>
Reads in n-gram count collections in the format that the Google n-grams Web1T
corpus comes in.
- Author:
- adampauls
-
Constructor Summary
ConstructorsConstructorDescriptionGoogleLmReader
(String rootDir, WordIndexer<W> wordIndexer, ConfigOptions opts) -
Method Summary
Modifier and TypeMethodDescriptionstatic <W> void
addToIndexer
(WordIndexer<W> wordIndexer, String sortedVocabPath) int
void
parse
(NgramOrderedLmReaderCallback<LongRef> callback)
-
Constructor Details
-
GoogleLmReader
-
-
Method Details
-
getLmOrder
public int getLmOrder() -
parse
- Specified by:
parse
in interfaceLmReader<LongRef,
NgramOrderedLmReaderCallback<LongRef>>
-
addToIndexer
- Parameters:
sortedVocabPath
-
-