Package opennlp.tools.tokenize

Schnittstelle Detokenizer

Alle bekannten Implementierungsklassen:

DictionaryDetokenizer

public interface Detokenizer

A Detokenizer merges tokens back to their detokenized representation.

Verschachtelte Klassen - Übersicht

Verschachtelte Klassen

Modifizierer und Typ	Schnittstelle	Beschreibung
static enum	Detokenizer.DetokenizationOperation	This enum contains an operation for every token to merge the tokens together to their detokenized form.

Methodenübersicht

Modifizierer und Typ	Methode	Beschreibung
Detokenizer.DetokenizationOperation[]	detokenize(String[] tokens)	Detokenizes the collection of tokens.
String	detokenize(String[] tokens, String splitMarker)	Detokenizes the input tokens into a String.

Methodendetails

detokenize

Detokenizer.DetokenizationOperation[] detokenize(String[] tokens)

Detokenizes the collection of tokens.

Parameter:

tokens - The elements which should be detokenized.

Gibt zurück:

The merge operations to handle given tokens.

detokenize

String detokenize(String[] tokens,
 String splitMarker)

Detokenizes the input tokens into a String. Tokens which are connected without a whitespace character in between can be separated by a given splitMarker.

Parameter:

tokens - The elements which should be concatenated.

splitMarker - The split marker or null.

Gibt zurück:

The concatenated tokens as a single string.