Class TokenizerStream

java.lang.Object

opennlp.tools.tokenize.TokenizerStream

All Implemented Interfaces:

AutoCloseable, opennlp.tools.util.ObjectStream<opennlp.tools.tokenize.TokenSample>

public class TokenizerStream
extends Object
implements opennlp.tools.util.ObjectStream<opennlp.tools.tokenize.TokenSample>

The TokenizerStream uses a Tokenizer to tokenize the input string and output samples.

Constructor Summary

Constructors

Constructor	Description
TokenizerStream(opennlp.tools.tokenize.Tokenizer tokenizer, opennlp.tools.util.ObjectStream<String> input)	Initializes a instance.

Method Summary

Modifier and Type	Method	Description
void	close()
opennlp.tools.tokenize.TokenSample	read()
void	reset()

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

TokenizerStream

public TokenizerStream(opennlp.tools.tokenize.Tokenizer tokenizer,
 opennlp.tools.util.ObjectStream<String> input)

Initializes a instance.

Parameters:

tokenizer - A working Tokenizer instance.

input - A plain text line stream.

Method Details

read

public opennlp.tools.tokenize.TokenSample read()
                                        throws IOException

Specified by:

read in interface opennlp.tools.util.ObjectStream<opennlp.tools.tokenize.TokenSample>

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface opennlp.tools.util.ObjectStream<opennlp.tools.tokenize.TokenSample>

Throws:

IOException

reset

public void reset()
           throws IOException,
UnsupportedOperationException

Specified by:

reset in interface opennlp.tools.util.ObjectStream<opennlp.tools.tokenize.TokenSample>

Throws:

IOException

UnsupportedOperationException