Package opennlp.tools.langdetect

Class LanguageDetectorME

java.lang.Object

opennlp.tools.langdetect.LanguageDetectorME

All Implemented Interfaces:

LanguageDetector

public class LanguageDetectorME
extends Object
implements LanguageDetector

Implements a learnable LanguageDetector.

This will process the entire string when called with predictLanguage(CharSequence) or predictLanguages(CharSequence).

If you want this to stop early, use probingPredictLanguages(CharSequence) or probingPredictLanguages(CharSequence, LanguageDetectorConfig). When run in probing mode, this starts at the beginning of the char sequence and runs language detection on chunks of text. If the end of the string is reached or there are LanguageDetectorConfig.getMinConsecImprovements() consecutive predictions for the best language and the confidence increases over those last predictions and if the difference in confidence between the highest confidence language and the second highest confidence language is greater than LanguageDetectorConfig.getMinDiff(), the language detector will stop and report the results.

The authors wish to thank Ken Krugler and Yalder} for the inspiration for many of the design components of this detector.

Constructor Summary

Constructors

Constructor	Description
LanguageDetectorME(LanguageDetectorModel model)	Initializes an instance with a specific LanguageDetectorModel.

Method Summary

Modifier and Type	Method	Description
String[]	getSupportedLanguages()
Language	predictLanguage(CharSequence content)	Predicts the Language for the full content length.
Language[]	predictLanguages(CharSequence content)	Predicts the languages for the full content length.
ProbingLanguageDetectionResult	probingPredictLanguages(CharSequence content)	This will stop processing early if the stopping criteria specified in LanguageDetectorConfig.DEFAULT_LANGUAGE_DETECTOR_CONFIG are met.
ProbingLanguageDetectionResult	probingPredictLanguages(CharSequence content, LanguageDetectorConfig config)	This will stop processing early if the stopping criteria specified in LanguageDetectorConfig.DEFAULT_LANGUAGE_DETECTOR_CONFIG are met.
static LanguageDetectorModel	train(ObjectStream<LanguageSample> samples, TrainingParameters mlParams, LanguageDetectorFactory factory)	Starts a training of a LanguageDetectorModel with the given parameters.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

LanguageDetectorME

public LanguageDetectorME(LanguageDetectorModel model)

Initializes an instance with a specific LanguageDetectorModel. Default feature generation is used.

Parameters:

model - the LanguageDetectorModel to be used.

Method Details

predictLanguages

public Language[] predictLanguages(CharSequence content)

Description copied from interface: LanguageDetector

Predicts the languages for the full content length.

Specified by:

predictLanguages in interface LanguageDetector

Parameters:

content - The textual content to detect potential languages from.

Returns:

the predicted languages

predictLanguage

public Language predictLanguage(CharSequence content)

Description copied from interface: LanguageDetector

Predicts the Language for the full content length.

Specified by:

predictLanguage in interface LanguageDetector

Parameters:

content - The textual content to detect potential languages from.

Returns:

the language with the highest confidence

getSupportedLanguages

public String[] getSupportedLanguages()

Specified by:

getSupportedLanguages in interface LanguageDetector

Returns:

Retrieves an array of language (codes) that are supported by a LanguageDetector.

probingPredictLanguages

public ProbingLanguageDetectionResult probingPredictLanguages(CharSequence content)

This will stop processing early if the stopping criteria specified in LanguageDetectorConfig.DEFAULT_LANGUAGE_DETECTOR_CONFIG are met.

Parameters:

content - content to be processed

Returns:

A computed ProbingLanguageDetectionResult.

probingPredictLanguages

public ProbingLanguageDetectionResult probingPredictLanguages(CharSequence content,
 LanguageDetectorConfig config)

This will stop processing early if the stopping criteria specified in LanguageDetectorConfig.DEFAULT_LANGUAGE_DETECTOR_CONFIG are met.

Parameters:

content - The textual content to process.

config - The LanguageDetectorConfig to customize detection.

Returns:

A computed ProbingLanguageDetectionResult.

train

public static LanguageDetectorModel train(ObjectStream<LanguageSample> samples,
 TrainingParameters mlParams,
 LanguageDetectorFactory factory)
                                   throws IOException

Starts a training of a LanguageDetectorModel with the given parameters.

Parameters:

samples - The ObjectStream of LanguageSample used as input for training.

mlParams - The TrainingParameters for the context of the training.

factory - The LanguageDetectorFactory for creating related objects defined via mlParams.

Returns:

A valid, trained LanguageDetectorModel instance.

Throws:

IOException - Thrown if IO errors occurred.