Package opennlp.tools.tokenize

Class DefaultTokenContextGenerator

java.lang.Object

opennlp.tools.tokenize.DefaultTokenContextGenerator

All Implemented Interfaces:

TokenContextGenerator

public class DefaultTokenContextGenerator
extends Object
implements TokenContextGenerator

A default TokenContextGenerator which produces events for maxent decisions for tokenization.

Constructor Summary

Constructors

Constructor	Description
DefaultTokenContextGenerator()	Initializes a plain DefaultTokenContextGenerator instance.
DefaultTokenContextGenerator(Set<String> inducedAbbreviations)	Initializes a customized DefaultTokenContextGenerator instance via a set of inducedAbbreviations.

Method Summary

Modifier and Type	Method	Description
String[]	getContext(String sentence, int index)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

DefaultTokenContextGenerator

public DefaultTokenContextGenerator()

Initializes a plain DefaultTokenContextGenerator instance.

DefaultTokenContextGenerator

public DefaultTokenContextGenerator(Set<String> inducedAbbreviations)

Initializes a customized DefaultTokenContextGenerator instance via a set of inducedAbbreviations.

Parameters:

inducedAbbreviations - The induced abbreviations to be used for this instance.

Method Details

getContext

public String[] getContext(String sentence,
 int index)

Specified by:

getContext in interface TokenContextGenerator

Parameters:

sentence - The string that represents a sentence.

index - The index to consider splitting tokens.

Returns:

An array of features for a sentence at the specified index.