opennlp.tools.formats

Class LeipzigDoccatSampleStream

java.lang.Object

opennlp.tools.util.FilterObjectStream<String,DocumentSample>

opennlp.tools.formats.LeipzigDoccatSampleStream

All Implemented Interfaces:

AutoCloseable, ObjectStream<DocumentSample>

public class LeipzigDoccatSampleStream
extends FilterObjectStream<String,DocumentSample>

Stream filter to produce document samples out of a Leipzig sentences.txt file. In the Leipzig corpus the encoding of the various sentences.txt file is defined by the language. The language must be specified to produce the category tags and is used to determine the correct input encoding.

The input text is tokenized with the SimpleTokenizer. The input text classified by the language model must also be tokenized by the SimpleTokenizer to produce exactly the same tokenization during testing and training.ø

Constructor Summary

Constructors

Constructor and Description
LeipzigDoccatSampleStream(String language, int sentencesPerDocument, InputStreamFactory in) Creates a new LeipzigDoccatSampleStream with the specified parameters.
LeipzigDoccatSampleStream(String language, int sentencesPerDocument, Tokenizer tokenizer, InputStreamFactory in) Creates a new LeipzigDoccatSampleStream with the specified parameters.

Method Summary

Modifier and Type	Method and Description
DocumentSample	read() Returns the next object.

Methods inherited from class opennlp.tools.util.FilterObjectStream

close, reset

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LeipzigDoccatSampleStream

public LeipzigDoccatSampleStream(String language,
                                 int sentencesPerDocument,
                                 Tokenizer tokenizer,
                                 InputStreamFactory in)
                          throws IOException

Creates a new LeipzigDoccatSampleStream with the specified parameters.

Parameters:

language - the Leipzig input sentences.txt file

sentencesPerDocument - the number of sentences which should be grouped into once DocumentSample

in - the InputStream pointing to the contents of the sentences.txt input file

Throws:

IOException - IOException

LeipzigDoccatSampleStream

public LeipzigDoccatSampleStream(String language,
                                 int sentencesPerDocument,
                                 InputStreamFactory in)
                          throws IOException

Creates a new LeipzigDoccatSampleStream with the specified parameters.

Parameters:

language - the Leipzig input sentences.txt file

sentencesPerDocument - the number of sentences which should be grouped into once DocumentSample

in - the InputStream pointing to the contents of the sentences.txt input file

Throws:

IOException - IOException

Method Detail

read

public DocumentSample read()
                    throws IOException

Description copied from interface: ObjectStream

Returns the next object. Calling this method repeatedly until it returns null will return each object from the underlying source exactly once.

Returns:

the next object or null to signal that the stream is exhausted

Throws:

IOException - if there is an error during reading