Class TwoPassDataIndexer

  • All Implemented Interfaces:
    DataIndexer

    public class TwoPassDataIndexer
    extends AbstractDataIndexer
    Collecting event and context counts by making two passes over the events. The first pass determines which contexts will be used by the model, and the second pass creates the events in memory containing only the contexts which will be used. This greatly reduces the amount of memory required for storing the events. During the first pass a temporary event file is created which is read during the second pass.
    • Constructor Detail

      • TwoPassDataIndexer

        public TwoPassDataIndexer()
    • Method Detail

      • index

        public void index​(ObjectStream<Event> eventStream)
                   throws IOException
        Description copied from interface: DataIndexer
        Performs the data indexing. Make sure the init(...) method is called first.
        Parameters:
        eventStream - a stream of events
        Throws:
        IOException