Package opennlp.tools.util.normalizer

Class LineBreakPreservingWhitespaceCharSequenceNormalizer

java.lang.Object
opennlp.tools.util.normalizer.LineBreakPreservingWhitespaceCharSequenceNormalizer
All Implemented Interfaces:
Serializable, opennlp.tools.util.normalizer.CharSequenceNormalizer, opennlp.tools.util.normalizer.OffsetAwareNormalizer
public class LineBreakPreservingWhitespaceCharSequenceNormalizer extends Object implements opennlp.tools.util.normalizer.OffsetAwareNormalizer
An OffsetAwareNormalizer that collapses runs of Unicode whitespace like WhitespaceCharSequenceNormalizer, but keeps line and paragraph structure: any whitespace run that contains a line break collapses to a single newline (U+000A) instead of a space, and leading and trailing whitespace is trimmed.

This is the form wanted for readable snippets and display: horizontal runs of spaces and tabs become a single space, yet a blank line between paragraphs survives as one newline rather than being flattened into the surrounding text. It reuses the cursor based CharClass.collapsePreserving(CharSequence, CodePointSet, int) engine, so it recognizes the full Unicode White_Space set with no regular expression.

See Also:

  • Constructor Details

    • LineBreakPreservingWhitespaceCharSequenceNormalizer

      public LineBreakPreservingWhitespaceCharSequenceNormalizer()

  • Method Details

    • getInstance

      public static LineBreakPreservingWhitespaceCharSequenceNormalizer getInstance()
      Returns the shared, stateless instance.
      Returns:
      the shared, stateless instance

    • normalize

      public CharSequence normalize(CharSequence text)
      Specified by:
      normalize in interface opennlp.tools.util.normalizer.CharSequenceNormalizer

    • normalizeAligned

      public opennlp.tools.util.normalizer.AlignedText normalizeAligned(CharSequence text)
      Specified by:
      normalizeAligned in interface opennlp.tools.util.normalizer.OffsetAwareNormalizer