opennlp.tools.tokenize.lang.en

Class TokenSampleStream

java.lang.Object

opennlp.tools.tokenize.lang.en.TokenSampleStream

All Implemented Interfaces:

Iterator<TokenSample>

public class TokenSampleStream
extends Object
implements Iterator<TokenSample>

Class which produces an Iterator<TokenSample> from a file of space delimited token. This class uses a number of English-specific heuristics to un-separate tokens which are typically found together in text.

Constructor Summary

Constructors

Constructor and Description
TokenSampleStream(InputStream is)

Method Summary

Methods

Modifier and Type	Method and Description
boolean	hasNext()
static void	main(String[] args)
TokenSample	next()
void	remove()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenSampleStream

public TokenSampleStream(InputStream is)
                  throws IOException

Throws:

IOException

Method Detail

hasNext

public boolean hasNext()

Specified by:

hasNext in interface Iterator<TokenSample>

next

public TokenSample next()

Specified by:

next in interface Iterator<TokenSample>

remove

public void remove()

Specified by:

remove in interface Iterator<TokenSample>

main

public static void main(String[] args)
                 throws IOException

Throws:

IOException