Package opennlp.tools.util.normalizer

Class WhitespaceCharSequenceNormalizer

java.lang.Object
opennlp.tools.util.normalizer.WhitespaceCharSequenceNormalizer
All Implemented Interfaces:
Serializable, opennlp.tools.util.normalizer.CharSequenceNormalizer, opennlp.tools.util.normalizer.OffsetAwareNormalizer
public class WhitespaceCharSequenceNormalizer extends Object implements opennlp.tools.util.normalizer.OffsetAwareNormalizer
A CharSequenceNormalizer that collapses each run of Unicode whitespace to a single ASCII space and trims the edges, reusing the cursor based CharClass.whitespace() engine.

Unlike a \s regular expression, this recognizes the full Unicode White_Space set (no-break space, ideographic space, the typographic spaces, line and paragraph separators, and so on), so spacing copied from the web, PDFs, or non-Latin sources normalizes consistently. It is the Unicode-aware, regex-free counterpart to ShrinkCharSequenceNormalizer.

See Also:

  • Constructor Details

    • WhitespaceCharSequenceNormalizer

      public WhitespaceCharSequenceNormalizer()

  • Method Details

    • getInstance

      public static WhitespaceCharSequenceNormalizer getInstance()
      Returns the shared, stateless instance.
      Returns:
      the shared, stateless instance

    • normalize

      public CharSequence normalize(CharSequence text)
      Specified by:
      normalize in interface opennlp.tools.util.normalizer.CharSequenceNormalizer

    • normalizeAligned

      public opennlp.tools.util.normalizer.AlignedText normalizeAligned(CharSequence text)
      Specified by:
      normalizeAligned in interface opennlp.tools.util.normalizer.OffsetAwareNormalizer