Package opennlp.tools.util.normalizer

Class UnicodeWhitespace

java.lang.Object

opennlp.tools.util.normalizer.UnicodeWhitespace

public final class UnicodeWhitespace
extends Object

Reference data for Unicode whitespace, plus O(1) membership lookups.

This is a static, immutable table of the 25 code points that carry the Unicode White_Space property, and the related 6 code points that are commonly mistaken for whitespace but carry White_Space=no (zero-width and other format characters). The data mirrors the tables in Whitespace character and the Unicode Character Database (PropList.txt).

The membership test is deliberately built from this explicit table rather than from Character.isWhitespace(int) or Character.isSpaceChar(int), both of which disagree with the Unicode White_Space property. Character.isWhitespace excludes the non-breaking spaces and NEL but includes the information-separator controls U+001C-U+001F; Character.isSpaceChar excludes tab, newline, and the other line breaks. isWhitespace(int) matches the standard exactly.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	UnicodeWhitespace.Breaking	Line-breaking behavior, mirroring the "Notes" column of the reference table.
static enum	UnicodeWhitespace.Category	Unicode general category for a whitespace or related code point.
static final record	UnicodeWhitespace.RelatedCharacter	One related code point that is commonly confused with whitespace but is not (White_Space=no).
static final record	UnicodeWhitespace.WhitespaceCharacter	One Unicode whitespace code point and its reference attributes.

Method Summary

Modifier and Type	Method	Description
static List<UnicodeWhitespace.WhitespaceCharacter>	all()	Returns the 25 Unicode whitespace characters, in ascending code point order.
static Optional<UnicodeWhitespace.WhitespaceCharacter>	byCodePoint(int codePoint)	Looks up the reference entry for a whitespace code point.
static int[]	codePoints()	Returns the whitespace code points, in ascending order.
static boolean	isLookalike(int codePoint)	Tests whether a code point is one of the related, non-whitespace look-alike format characters.
static boolean	isWhitespace(int codePoint)	Tests whether a code point carries the Unicode White_Space property.
static List<UnicodeWhitespace.WhitespaceCharacter>	lineBreaks()	Returns the whitespace characters that force a line or paragraph break.
static List<UnicodeWhitespace.RelatedCharacter>	lookalikes()	Returns the related, non-whitespace look-alike format characters.
static List<UnicodeWhitespace.WhitespaceCharacter>	nonBreaking()	Returns the non-breaking whitespace characters.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

isWhitespace

public static boolean isWhitespace(int codePoint)

Tests whether a code point carries the Unicode White_Space property.

Parameters:

codePoint - The code point to test. Out-of-range values (negative or beyond Character.MAX_CODE_POINT) simply return false.

Returns:

true if the code point is one of the 25 Unicode whitespace characters.

isLookalike

public static boolean isLookalike(int codePoint)

Tests whether a code point is one of the related, non-whitespace look-alike format characters.

Parameters:

codePoint - The code point to test.

Returns:

true if the code point is in the look-alike set.

byCodePoint

public static Optional<UnicodeWhitespace.WhitespaceCharacter> byCodePoint(int codePoint)

Looks up the reference entry for a whitespace code point.

Parameters:

codePoint - The code point.

Returns:

The UnicodeWhitespace.WhitespaceCharacter, or Optional.empty() if it is not whitespace.

all

public static List<UnicodeWhitespace.WhitespaceCharacter> all()

Returns the 25 Unicode whitespace characters, in ascending code point order.

Returns:

the 25 Unicode whitespace characters, in ascending code point order

lookalikes

public static List<UnicodeWhitespace.RelatedCharacter> lookalikes()

Returns the related, non-whitespace look-alike format characters.

Returns:

the related, non-whitespace look-alike format characters

lineBreaks

public static List<UnicodeWhitespace.WhitespaceCharacter> lineBreaks()

Returns the whitespace characters that force a line or paragraph break.

Returns:

the whitespace characters that force a line or paragraph break

nonBreaking

public static List<UnicodeWhitespace.WhitespaceCharacter> nonBreaking()

Returns the non-breaking whitespace characters.

Returns:

the non-breaking whitespace characters

codePoints

public static int[] codePoints()

Returns the whitespace code points, in ascending order.

Returns:

the whitespace code points, in ascending order