Package opennlp.tools.tokenize.uax29

Interface WordTokenizer.TokenHandler

Enclosing class:
WordTokenizer
Functional Interface:
This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.
@FunctionalInterface public static interface WordTokenizer.TokenHandler
Receives each word token as a character range and its type, with no allocation.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    token(int start, int end, WordType type)
    Accepts one word token.

  • Method Details

    • token

      void token(int start, int end, WordType type)
      Accepts one word token.
      Parameters:
      start - The inclusive start character offset.
      end - The exclusive end character offset.
      type - The token category.