Package opennlp.tools.tokenize
Class DefaultTokenContextGenerator
- java.lang.Object
-
- opennlp.tools.tokenize.DefaultTokenContextGenerator
-
- All Implemented Interfaces:
TokenContextGenerator
public class DefaultTokenContextGenerator extends Object implements TokenContextGenerator
A defaultTokenContextGenerator
which produces events for maxent decisions for tokenization.
-
-
Constructor Summary
Constructors Constructor Description DefaultTokenContextGenerator()
Initializes a plainDefaultTokenContextGenerator
instance.DefaultTokenContextGenerator(Set<String> inducedAbbreviations)
Initializes a customizedDefaultTokenContextGenerator
instance via a set ofinducedAbbreviations
.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description String[]
getContext(String sentence, int index)
-
-
-
Constructor Detail
-
DefaultTokenContextGenerator
public DefaultTokenContextGenerator()
Initializes a plainDefaultTokenContextGenerator
instance.
-
DefaultTokenContextGenerator
public DefaultTokenContextGenerator(Set<String> inducedAbbreviations)
Initializes a customizedDefaultTokenContextGenerator
instance via a set ofinducedAbbreviations
.- Parameters:
inducedAbbreviations
- The induced abbreviations to be used for this instance.
-
-
Method Detail
-
getContext
public String[] getContext(String sentence, int index)
- Specified by:
getContext
in interfaceTokenContextGenerator
- Parameters:
sentence
- The string that represents a sentence.index
- The index to consider splitting tokens.- Returns:
- An array of features for a
sentence
at the specifiedindex
.
-
-