public class TokenSampleStream extends FilterObjectStream<String,TokenSample>
TokenSample
s out of them. The input string sample is tokenized if a
whitespace or the special separator chars occur.
Sample:
"token1 token2 token3<SPLIT>token4"
The tokens token1 and token2 are separated by a whitespace, token3 and token3
are separated by the special character sequence, in this case the default
split sequence.
The sequence must be unique in the input string and is not escaped.
Constructor and Description |
---|
TokenSampleStream(ObjectStream<String> sentences) |
TokenSampleStream(ObjectStream<String> sampleStrings,
String separatorChars) |
Modifier and Type | Method and Description |
---|---|
TokenSample |
read()
Returns the next object.
|
close, reset
public TokenSampleStream(ObjectStream<String> sampleStrings, String separatorChars)
public TokenSampleStream(ObjectStream<String> sentences)
public TokenSample read() throws IOException
ObjectStream
IOException
- if there is an error during readingCopyright © 2018 The Apache Software Foundation. All rights reserved.