TokenSampleStream (Apache OpenNLP Tools 2.1.1 API)

java.lang.Object
- opennlp.tools.util.FilterObjectStream<String,TokenSample>
- - opennlp.tools.tokenize.TokenSampleStream

All Implemented Interfaces:

AutoCloseable, ObjectStream<TokenSample>
```
public class TokenSampleStream
extends FilterObjectStream<String,TokenSample>
```
This class is a stream filter which reads in string encoded samples and creates samples out of them. The input string sample is tokenized if a whitespace or the special separator chars occur.
Sample:
"token1 token2 token3<SPLIT>token4"
The tokens token1 and token2 are separated by a whitespace, token3 and token4 are separated by the special character sequence. In this case, the default split sequence applies.
Note: The sequence must be unique in the input string and is not escaped.

Constructor Summary

Constructors
Constructor	Description
`TokenSampleStream(ObjectStream<String> sentences)`	Initializes a `instance`.
`TokenSampleStream(ObjectStream<String> samples, String separatorChars)`	Initializes a `instance`.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type Method Description

TokenSample read()
Returns the next ObjectStream object.
- Methods inherited from class opennlp.tools.util.FilterObjectStream
  close, reset
- Methods inherited from class java.lang.Object
  equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - TokenSampleStream
```
public TokenSampleStream(ObjectStream<String> samples,
                         String separatorChars)
```
    Initializes a instance.
    
    Parameters:
    
    samples - A plain text line stream. Must not be null.
    
    separatorChars - The characters to be considered separators. See TokenSample.DEFAULT_SEPARATOR_CHARS. Must not be null.
  - TokenSampleStream
```
public TokenSampleStream(ObjectStream<String> sentences)
```
    Initializes a instance.
    
    Parameters:
    
    sentences - A plain text line stream. Must not be null.
- Method Detail
  - read
```
public TokenSample read()
                 throws IOException
```
    Description copied from interface: ObjectStream
    
    Returns the next ObjectStream object. Calling this method repeatedly until it returns null will return each object from the underlying source exactly once.
    
    Returns:
    
    The next object or null to signal that the stream is exhausted.
    
    Throws:
    
    IOException - Thrown if there is an error during reading.