opennlp.tools.ml.model

Class TwoPassDataIndexer

java.lang.Object

opennlp.tools.ml.model.AbstractDataIndexer

opennlp.tools.ml.model.TwoPassDataIndexer

All Implemented Interfaces:

DataIndexer

public class TwoPassDataIndexer
extends AbstractDataIndexer

Collecting event and context counts by making two passes over the events. The first pass determines which contexts will be used by the model, and the second pass creates the events in memory containing only the contexts which will be used. This greatly reduces the amount of memory required for storing the events. During the first pass a temporary event file is created which is read during the second pass.

Field Summary

Fields inherited from class opennlp.tools.ml.model.AbstractDataIndexer

CUTOFF_DEFAULT, CUTOFF_PARAM, SORT_DEFAULT, SORT_PARAM

Constructor Summary

Constructors

Constructor and Description
TwoPassDataIndexer()

Method Summary

Modifier and Type	Method and Description
void	index(ObjectStream<Event> eventStream) Performs the data indexing.

Methods inherited from class opennlp.tools.ml.model.AbstractDataIndexer

getContexts, getNumEvents, getNumTimesEventsSeen, getOutcomeLabels, getOutcomeList, getPredCounts, getPredLabels, getValues, init

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TwoPassDataIndexer

public TwoPassDataIndexer()

Method Detail

index

public void index(ObjectStream<Event> eventStream)
           throws IOException

Description copied from interface: DataIndexer

Performs the data indexing. Make sure the init(...) method is called first.

Parameters:

eventStream - a stream of events

Throws:

IOException