Package opennlp.tools.util.normalizer

Class UrlCharSequenceNormalizer

java.lang.Object

opennlp.tools.util.normalizer.UrlCharSequenceNormalizer

All Implemented Interfaces:

Serializable, opennlp.tools.util.normalizer.CharSequenceNormalizer

public class UrlCharSequenceNormalizer
extends Object
implements opennlp.tools.util.normalizer.CharSequenceNormalizer

A CharSequenceNormalizer implementation that normalizes text in terms of URls and email addresses. Every encounter will be replaced by a whitespace.

Normalization runs in two passes:

1. URLs: a lowercase http:// or https:// scheme followed by at least one character out of the body set [-_.?&~;+=/#0-9A-Za-z] becomes one space; the match ends at the first character outside that set, so a colon (port), a percent escape, an at sign (userinfo), or a non-ASCII label cuts it short.
2. Email addresses: a maximal run of the local-part set [-+_.0-9A-Za-z] whose left neighbor is outside that set, an @, and a domain run out of [-.0-9A-Za-z] that must not start with a dot and must span at least two chars, become one space.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
UrlCharSequenceNormalizer()

Method Summary

Modifier and Type	Method	Description
static UrlCharSequenceNormalizer	getInstance()	Returns the shared, stateless instance.
CharSequence	normalize(CharSequence text)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

UrlCharSequenceNormalizer

public UrlCharSequenceNormalizer()

Method Details

getInstance

public static UrlCharSequenceNormalizer getInstance()

Returns the shared, stateless instance.

Returns:

the shared, stateless instance

normalize

public CharSequence normalize(CharSequence text)

Specified by:

normalize in interface opennlp.tools.util.normalizer.CharSequenceNormalizer