Package edu.berkeley.nlp.lm.io
Class TextReader<W>
java.lang.Object
edu.berkeley.nlp.lm.io.TextReader<W>
- Type Parameters:
W
-
- All Implemented Interfaces:
LmReader<LongRef,
LmReaderCallback<LongRef>>
Class for reading raw text files.
- Author:
- adampauls
-
Constructor Summary
ConstructorsConstructorDescriptionTextReader
(Iterable<String> lineIterator, WordIndexer<W> wordIndexer) TextReader
(List<String> inputFiles, WordIndexer<W> wordIndexer) -
Method Summary
Modifier and TypeMethodDescriptionvoid
parse
(LmReaderCallback<LongRef> callback) Reads newline-separated plain text from inputFiles, and writes an ARPA lm file to outputFile.
-
Constructor Details
-
TextReader
-
TextReader
-
-
Method Details
-
parse
Reads newline-separated plain text from inputFiles, and writes an ARPA lm file to outputFile. If files have a .gz suffix, then they will be (un)zipped as necessary.- Specified by:
parse
in interfaceLmReader<LongRef,
LmReaderCallback<LongRef>> - Parameters:
inputFiles
-outputFile
-
-