Package opennlp.tools.tokenize

Class TokSpanEventStream

java.lang.Object

opennlp.tools.util.AbstractEventStream<TokenSample>

opennlp.tools.tokenize.TokSpanEventStream

All Implemented Interfaces:

AutoCloseable, ObjectStream<Event>

public class TokSpanEventStream
extends AbstractEventStream<TokenSample>

This class reads the samples via an Iterator and converts the samples into events which can be used by the maxent library for training.

Constructor Summary

Constructors

Constructor	Description
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics)	Initializes a new event stream based on the data stream using a TokenContextGenerator that relies on a DefaultTokenContextGenerator.
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg)	Initializes a new event stream based on the data stream using a TokenContextGenerator.
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg)	Initializes a new event stream based on the data stream using a TokenContextGenerator.

Method Summary

Methods inherited from class opennlp.tools.util.AbstractEventStream

close, read, reset

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokSpanEventStream

public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
                          boolean skipAlphaNumerics,
                          Pattern alphaNumeric,
                          TokenContextGenerator cg)

Initializes a new event stream based on the data stream using a TokenContextGenerator.

Parameters:

tokenSamples - The data stream for this event stream.

skipAlphaNumerics - Whether alphanumerics are skipped, or not.

alphaNumeric - A custom alphanumeric Pattern or null. Default is: "^[A-Za-z0-9]+$", provided by Factory.DEFAULT_ALPHANUMERIC.

cg - A TokenContextGenerator which should be used for the event stream d.

TokSpanEventStream

public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
                          boolean skipAlphaNumerics,
                          TokenContextGenerator cg)

Initializes a new event stream based on the data stream using a TokenContextGenerator.

Parameters:

tokenSamples - The data stream for this event stream.

skipAlphaNumerics - Whether alphanumerics are skipped, or not.

cg - A TokenContextGenerator which should be used for the event stream d.

TokSpanEventStream

public TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
                          boolean skipAlphaNumerics)

Initializes a new event stream based on the data stream using a TokenContextGenerator that relies on a DefaultTokenContextGenerator.

Parameters:

tokenSamples - The data stream for this event stream.

skipAlphaNumerics - Whether alphanumerics are skipped, or not.