Package opennlp.tools.util.normalizer
package opennlp.tools.util.normalizer
-
ClassDescriptionThe result of a normalization that keeps the original text alongside the normalized form and a full
Alignmentbetween them.A bidirectional alignment between an original text and a normalized form of it.Builds anAlignmentas the normalized text is produced, by recording each edit in order.A configurable class of Unicode code points and the cursor based operations over it.A char sequence normalizer, used to adjusting (prune, substitute, add, etc.)An immutable set of Unicode code points with O(1) membership.ACharSequenceNormalizerthat can additionally report theAlignmentfrom its normalized output back to the input, so a span found in the normalized text maps to the exact character offsets of the original.Reference data for Unicode dashes, plus O(1) membership lookups.The Unicode general category of a dash code point.One Unicode dash code point and its reference attributes.Reference data for Unicode whitespace, plus O(1) membership lookups.Line-breaking behavior, mirroring the "Notes" column of the reference table.Unicode general category for a whitespace or related code point.One related code point that is commonly confused with whitespace but is not (White_Space=no).One Unicode whitespace code point and its reference attributes.