Package opennlp.tools.util.normalizer

Class NormalizationProfiles

java.lang.Object
opennlp.tools.util.normalizer.NormalizationProfiles
public final class NormalizationProfiles extends Object
A registry of NormalizationProfiles by language, with detection-based fallback. This is the language dispatch the design note calls for: pick the profile for a requested language, or detect the language with a LanguageDetector when it is unspecified. The covered languages are the Snowball stemmer algorithms that name a natural language -- every SnowballStemmer.ALGORITHM except PORTER, which is an English-only algorithm variant rather than a distinct language. Several codes can map to one algorithm (the three Norwegian written standards all use NORWEGIAN).

Profiles are keyed by ISO 639-3 code (what LanguageDetector produces); forLanguage(String) also accepts ISO 639-1 two-letter codes.

  • Method Details

    • forLanguage

      public static Optional<NormalizationProfile> forLanguage(String language)
      Returns the profile for a language.
      Parameters:
      language - An ISO 639-3 or ISO 639-1 language code; case-insensitive. Must not be null.
      Returns:
      The profile, or empty if the language has no Snowball stemmer.
      Throws:
      IllegalArgumentException - if language is null.

    • detect

      public static Optional<NormalizationProfile> detect(CharSequence text, opennlp.tools.langdetect.LanguageDetector detector)
      Detects the language of text and returns its profile.
      Parameters:
      text - The text to detect. Must not be null.
      detector - The language detector to use. Must not be null.
      Returns:
      The profile for the detected language, or empty if it has no Snowball stemmer.
      Throws:
      IllegalArgumentException - if text or detector is null.

    • supportedLanguages

      public static Set<String> supportedLanguages()
      Returns the ISO 639-3 codes of the supported languages.
      Returns:
      the ISO 639-3 codes of the supported languages