Package opennlp.tools.util.normalizer

Class Confusables

java.lang.Object
opennlp.tools.util.normalizer.Confusables
public final class Confusables extends Object
Computes the Unicode confusable skeleton of text, following the skeleton algorithm defined in UTS #39 (Unicode Security Mechanisms). Two strings are confusable, for example Latin "paypal" and a version using Cyrillic lookalikes, exactly when their skeletons are equal.

The mapping is loaded once from the confusables.txt resource of the Unicode security data. The skeleton of a string is NFD(map(NFD(s))): decompose, replace each code point with its prototype, and decompose again. This changes length and offsets, so it is a matching-only comparison form, not an offset-preserving transform.

This implements only the UTS #39 skeleton transform and the confusable-detection test built on skeleton equality. The other mechanisms defined in the report, such as identifier restriction levels, mixed-script and whole-script confusable detection, and the bidirectional skeleton, are out of scope.

  • Method Details

    • skeleton

      public static String skeleton(CharSequence text)
      Returns the confusable skeleton of text: NFD(map(NFD(text))) where map replaces each code point with its UTS #39 prototype. The skeleton is for comparison only; it is not human-readable text and does not preserve offsets.
      Parameters:
      text - The text to reduce.
      Returns:
      The skeleton.
      Throws:
      IllegalArgumentException - Thrown if text is null.

    • confusable

      public static boolean confusable(CharSequence left, CharSequence right)
      Returns whether left and right are confusable. They are confusable when their skeletons are equal.
      Parameters:
      left - The first string.
      right - The second string.
      Returns:
      whether left and right are confusable
      Throws:
      IllegalArgumentException - Thrown if left or right is null.