Package opennlp.uima.tokenize

Class Tokenizer

java.lang.Object
org.apache.uima.analysis_component.AnalysisComponent_ImplBase
org.apache.uima.analysis_component.Annotator_ImplBase
org.apache.uima.analysis_component.CasAnnotator_ImplBase
opennlp.uima.tokenize.AbstractTokenizer
opennlp.uima.tokenize.Tokenizer
All Implemented Interfaces:
org.apache.uima.analysis_component.AnalysisComponent
public final class Tokenizer extends AbstractTokenizer
OpenNLP Tokenizer annotator.

Mandatory parameters

Type Name Description
String opennlp.uima.ModelName The name of the model file
String opennlp.uima.SentenceType The full name of the sentence type
String opennlp.uima.TokenType The full name of the token type

Optional parameters

Type Name Description
String opennlp.uima.ProbabilityFeature The name of the double probability feature (not set by default)
See Also:

  • Constructor Summary

    Constructors
    Constructor
    Description
    Tokenizer()
    Initializes a new instance.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    destroy()
    Releases allocated resources.
    void
    initialize(org.apache.uima.UimaContext context)
    Initializes the current instance with the given context.
    void
    typeSystemInit(org.apache.uima.cas.TypeSystem typeSystem)
    Initializes the type system.

    Methods inherited from class opennlp.uima.tokenize.AbstractTokenizer

    process

    Methods inherited from class org.apache.uima.analysis_component.CasAnnotator_ImplBase

    getRequiredCasInterface, process

    Methods inherited from class org.apache.uima.analysis_component.Annotator_ImplBase

    getCasInstancesRequired, hasNext, next

    Methods inherited from class org.apache.uima.analysis_component.AnalysisComponent_ImplBase

    batchProcessComplete, collectionProcessComplete, reconfigure, setResultSpecification

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • Tokenizer

      public Tokenizer()
      Initializes a new instance.

      Note: Use initialize(UimaContext) to initialize this instance. Not use the constructor.

  • Method Details

    • initialize

      public void initialize(org.apache.uima.UimaContext context) throws org.apache.uima.resource.ResourceInitializationException
      Initializes the current instance with the given context.

      Note: Do all initialization in this method, do not use the constructor.

      Specified by:
      initialize in interface org.apache.uima.analysis_component.AnalysisComponent
      Overrides:
      initialize in class AbstractTokenizer
      Throws:
      org.apache.uima.resource.ResourceInitializationException

    • typeSystemInit

      public void typeSystemInit(org.apache.uima.cas.TypeSystem typeSystem) throws org.apache.uima.analysis_engine.AnalysisEngineProcessException
      Initializes the type system.
      Overrides:
      typeSystemInit in class AbstractTokenizer
      Throws:
      org.apache.uima.analysis_engine.AnalysisEngineProcessException

    • destroy

      public void destroy()
      Releases allocated resources.
      Specified by:
      destroy in interface org.apache.uima.analysis_component.AnalysisComponent
      Overrides:
      destroy in class org.apache.uima.analysis_component.AnalysisComponent_ImplBase