Package opennlp.tools.tokenize

Class WhitespaceTokenStream

java.lang.Object

opennlp.tools.util.FilterObjectStream<TokenSample,String>

opennlp.tools.tokenize.WhitespaceTokenStream

All Implemented Interfaces:

AutoCloseable, ObjectStream<String>

public class WhitespaceTokenStream
extends FilterObjectStream<TokenSample,String>

This stream formats ObjectStream of samples into whitespace separated token strings.

Constructor Summary

Constructors

Constructor	Description
WhitespaceTokenStream(ObjectStream<TokenSample> tokens)	Initializes a WhitespaceTokenStream.

Method Summary

Modifier and Type	Method	Description
String	read()	Returns the next ObjectStream object.

Methods inherited from class opennlp.tools.util.FilterObjectStream

close, reset

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

WhitespaceTokenStream

public WhitespaceTokenStream(ObjectStream<TokenSample> tokens)

Initializes a WhitespaceTokenStream.

Parameters:

tokens - The stream of tokens to be separated.

Method Details

read

public String read()
            throws IOException

Description copied from interface: ObjectStream

Returns the next ObjectStream object. Calling this method repeatedly until it returns null will return each object from the underlying source exactly once.

Returns:

The next object or null to signal that the stream is exhausted.

Throws:

IOException - Thrown if there is an error during reading.