Class TokenSampleStream

  • All Implemented Interfaces:
    Iterator<TokenSample>

    public class TokenSampleStream
    extends Object
    implements Iterator<TokenSample>
    Class which produces an Iterator<TokenSample> from a file of space delimited token. This class uses a number of English-specific heuristics to un-separate tokens which are typically found together in text.