Package opennlp.tools.util.normalizer

Class DigitCharSequenceNormalizer

java.lang.Object
opennlp.tools.util.normalizer.DigitCharSequenceNormalizer
All Implemented Interfaces:
Serializable, opennlp.tools.util.normalizer.CharSequenceNormalizer, opennlp.tools.util.normalizer.OffsetAwareNormalizer
public class DigitCharSequenceNormalizer extends Object implements opennlp.tools.util.normalizer.OffsetAwareNormalizer
A CharSequenceNormalizer that maps Unicode decimal digits to their ASCII equivalents, so for example Arabic-Indic, Devanagari, or fullwidth digits all become 0-9.

It maps a code point when Character.digit(int, int) reports a value of 0- 9 in radix ten, that is, when the code point is a Unicode decimal digit. Other numeric forms (Roman numerals, superscripts, circled numbers, fractions) are not decimal digits and are left unchanged. Scanning is a single O(1)-per-code-point cursor pass with no regular expression.

See Also:

  • Constructor Details

    • DigitCharSequenceNormalizer

      public DigitCharSequenceNormalizer()

  • Method Details

    • getInstance

      public static DigitCharSequenceNormalizer getInstance()
      Returns the shared, stateless instance.
      Returns:
      the shared, stateless instance

    • normalize

      public CharSequence normalize(CharSequence text)
      Specified by:
      normalize in interface opennlp.tools.util.normalizer.CharSequenceNormalizer

    • normalizeAligned

      public opennlp.tools.util.normalizer.AlignedText normalizeAligned(CharSequence text)
      Specified by:
      normalizeAligned in interface opennlp.tools.util.normalizer.OffsetAwareNormalizer