Package opennlp.tools.tokenize

Klasse TokenSampleStream

java.lang.Object

opennlp.tools.util.FilterObjectStream<String,TokenSample>

opennlp.tools.tokenize.TokenSampleStream

Alle implementierten Schnittstellen:

AutoCloseable, ObjectStream<TokenSample>

public class TokenSampleStream
extends FilterObjectStream<String,TokenSample>

This class is a stream filter which reads in string encoded samples and creates samples out of them. The input string sample is tokenized if a whitespace or the special separator chars occur.

Sample:
"token1 token2 token3<SPLIT>token4"
The tokens token1 and token2 are separated by a whitespace, token3 and token4 are separated by the special character sequence. In this case, the default split sequence applies.

Note: The sequence must be unique in the input string and is not escaped.

Konstruktorübersicht

Konstruktoren

Konstruktor	Beschreibung
TokenSampleStream(ObjectStream<String> sentences)	Initializes a instance.
TokenSampleStream(ObjectStream<String> samples, String separatorChars)	Initializes a instance.

Methodenübersicht

Modifizierer und Typ	Methode	Beschreibung
TokenSample	read()	Returns the next ObjectStream object.

Von Klasse geerbte Methoden opennlp.tools.util.FilterObjectStream

close, reset

Von Klasse geerbte Methoden java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Konstruktordetails

TokenSampleStream

public TokenSampleStream(ObjectStream<String> samples,
 String separatorChars)

Initializes a instance.

Parameter:

samples - A plain text line stream. Must not be null.

separatorChars - The characters to be considered separators. See TokenSample.DEFAULT_SEPARATOR_CHARS. Must not be null.

TokenSampleStream

public TokenSampleStream(ObjectStream<String> sentences)

Initializes a instance.

Parameter:

sentences - A plain text line stream. Must not be null.

Methodendetails

read

public TokenSample read()
                 throws IOException

Beschreibung aus Schnittstelle kopiert: ObjectStream

Returns the next ObjectStream object. Calling this method repeatedly until it returns null will return each object from the underlying source exactly once.

Gibt zurück:

The next object or null to signal that the stream is exhausted.

Löst aus:

IOException - Thrown if there is an error during reading.