opennlp.tools.tokenize.lang.en

Class TokenSampleStream

java.lang.Object

opennlp.tools.tokenize.lang.en.TokenSampleStream

All Implemented Interfaces:

Iterator<TokenSample>

public class TokenSampleStream
extends Object
implements Iterator<TokenSample>

Class which produces an Iterator<TokenSample> from a file of space delimited token. This class uses a number of English-specific heuristics to un-separate tokens which are typically found together in text.

Constructor Summary

Constructors

Constructor and Description
TokenSampleStream(InputStream is)

Method Summary

Modifier and Type	Method and Description
boolean	hasNext()
TokenSample	next()
void	remove()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Constructor Detail

TokenSampleStream

public TokenSampleStream(InputStream is)
                  throws IOException

Throws:

IOException

Method Detail

hasNext

public boolean hasNext()

Specified by:

hasNext in interface Iterator<TokenSample>

next

public TokenSample next()

Specified by:

next in interface Iterator<TokenSample>

remove

public void remove()

Specified by:

remove in interface Iterator<TokenSample>