Detokenizer (Apache OpenNLP Tools 2.1.1 API)

All Known Implementing Classes:

DictionaryDetokenizer
```
public interface Detokenizer
```
A Detokenizer merges tokens back to their detokenized representation.

Nested Class Summary

Nested Classes
Modifier and Type	Interface	Description
`static class`	`Detokenizer.DetokenizationOperation`	This enum contains an operation for every token to merge the tokens together to their detokenized form.

Method Summary

All Methods Instance Methods Abstract Methods
Modifier and Type	Method	Description
`Detokenizer.DetokenizationOperation[]`	`detokenize(String[] tokens)`	Detokenizes the collection of tokens.
`String`	`detokenize(String[] tokens, String splitMarker)`	Detokenizes the input `tokens` into a String.

- Method Detail
  - detokenize
```
Detokenizer.DetokenizationOperation[] detokenize(String[] tokens)
```
    Detokenizes the collection of tokens.
    
    Parameters:
    
    tokens - The elements which should be detokenized.
    
    Returns:
    
    The merge operations to handle given tokens.
  - detokenize
```
String detokenize(String[] tokens,
                  String splitMarker)
```
    Detokenizes the input tokens into a String. Tokens which are connected without a whitespace character in between can be separated by a given splitMarker.
    
    Parameters:
    
    tokens - The elements which should be concatenated.
    
    splitMarker - The split marker or null.
    
    Returns:
    
    The concatenated tokens as a single string.