Detokenizer (Apache OpenNLP Tools 2.1.0 API)

All Known Implementing Classes:

DictionaryDetokenizer
```
public interface Detokenizer
```
A Detokenizer merges tokens back to their untokenized representation.

Nested Class Summary

Nested Classes
Modifier and Type	Interface	Description
`static class`	`Detokenizer.DetokenizationOperation`	This enum contains an operation for every token to merge the tokens together to their detokenized form.

Method Summary

All Methods Instance Methods Abstract Methods
Modifier and Type	Method	Description
`Detokenizer.DetokenizationOperation[]`	`detokenize(String[] tokens)`	Detokenize the input tokens.
`String`	`detokenize(String[] tokens, String splitMarker)`	Detokenize the input tokens into a String.

- Method Detail
  - detokenize
```
Detokenizer.DetokenizationOperation[] detokenize(String[] tokens)
```
    Detokenize the input tokens.
    
    Parameters:
    
    tokens - the tokens to detokenize.
    
    Returns:
    
    the merge operations to detokenize the input tokens.
  - detokenize
```
String detokenize(String[] tokens,
                  String splitMarker)
```
    Detokenize the input tokens into a String. Tokens which are connected without a space inbetween can be separated by a split marker.
    
    Parameters:
    
    tokens - the token which should be concatenated
    
    splitMarker - the split marker or null
    
    Returns:
    
    the concatenated tokens