Class TokenPatternFeatureGenerator

java.lang.Object

opennlp.tools.util.featuregen.TokenPatternFeatureGenerator

All Implemented Interfaces:

opennlp.tools.util.featuregen.AdaptiveFeatureGenerator

public class TokenPatternFeatureGenerator
extends Object
implements opennlp.tools.util.featuregen.AdaptiveFeatureGenerator

Partitions tokens into sub-tokens based on character classes and generates class features for each of the sub-tokens and combinations of those sub-tokens.

See Also:

• AdaptiveFeatureGenerator
• Tokenizer

Constructor Summary

Constructors

Constructor	Description
TokenPatternFeatureGenerator()	Initializes a TokenPatternFeatureGenerator.
TokenPatternFeatureGenerator(opennlp.tools.tokenize.Tokenizer supportTokenizer)	Initializes a TokenPatternFeatureGenerator instance.

Method Summary

Modifier and Type	Method	Description
void	createFeatures(List<String> feats, String[] toks, int index, String[] preds)

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface opennlp.tools.util.featuregen.AdaptiveFeatureGenerator

clearAdaptiveData, updateAdaptiveData

Constructor Details

TokenPatternFeatureGenerator

public TokenPatternFeatureGenerator()

Initializes a TokenPatternFeatureGenerator. For tokenization the SimpleTokenizer is used.

TokenPatternFeatureGenerator

public TokenPatternFeatureGenerator(opennlp.tools.tokenize.Tokenizer supportTokenizer)

Initializes a TokenPatternFeatureGenerator instance.

Parameters:

supportTokenizer - The Tokenizer to be used.

Method Details

createFeatures

public void createFeatures(List<String> feats,
 String[] toks,
 int index,
 String[] preds)

Specified by:

createFeatures in interface opennlp.tools.util.featuregen.AdaptiveFeatureGenerator