Package opennlp.tools.util.normalizer

Class EmoticonToEmojiCharSequenceNormalizer

java.lang.Object
opennlp.tools.util.normalizer.EmoticonToEmojiCharSequenceNormalizer
All Implemented Interfaces:
Serializable, opennlp.tools.util.normalizer.CharSequenceNormalizer, opennlp.tools.util.normalizer.OffsetAwareNormalizer
public final class EmoticonToEmojiCharSequenceNormalizer extends Object implements opennlp.tools.util.normalizer.OffsetAwareNormalizer
A CharSequenceNormalizer that folds ASCII emoticons to emoji, using the bundled emoji-emoticons.txt mapping (for example :-) to U+1F642 SLIGHTLY SMILING FACE).

An emoticon folds only when it stands alone as a whitespace-delimited unit (the text boundary or Unicode White_Space on both sides), so sequences inside ordinary text such as the :/ in https:// are never touched. Matching is longest first at each position. Apply this normalizer before tokenization if emoticons should survive as single tokens. This is an offset-changing transform: normalizeAligned(CharSequence) reports the Alignment from the folded text back to the input. The reverse direction is EmojiToEmoticonCharSequenceNormalizer.

See Also:

  • Method Details

    • getInstance

      public static EmoticonToEmojiCharSequenceNormalizer getInstance()
      Returns the shared, stateless instance.
      Returns:
      the shared, stateless instance

    • normalize

      public CharSequence normalize(CharSequence text)
      Specified by:
      normalize in interface opennlp.tools.util.normalizer.CharSequenceNormalizer

    • normalizeAligned

      public opennlp.tools.util.normalizer.AlignedText normalizeAligned(CharSequence text)
      Specified by:
      normalizeAligned in interface opennlp.tools.util.normalizer.OffsetAwareNormalizer