Class DefaultTokenContextGenerator

    • Constructor Detail

      • DefaultTokenContextGenerator

        public DefaultTokenContextGenerator()
        Creates a default context generator for tokenizer.
      • DefaultTokenContextGenerator

        public DefaultTokenContextGenerator​(Set<String> inducedAbbreviations)
        Creates a default context generator for tokenizer.
        Parameters:
        inducedAbbreviations - the induced abbreviations
    • Method Detail

      • getContext

        public String[] getContext​(String sentence,
                                   int index)
        Description copied from interface: TokenContextGenerator
        Returns an array of features for the specified sentence string at the specified index.
        Specified by:
        getContext in interface TokenContextGenerator
        Parameters:
        sentence - The string for a sentence.
        index - The index to consider splitting as a token.
        Returns:
        an array of features for the specified sentence string at the specified index.