Interface LanguageModel

  • All Known Implementing Classes:
    NGramLanguageModel

    public interface LanguageModel
    A language model can calculate the probability p (between 0 and 1) of a certain sequence of tokens, given its underlying vocabulary.
    • Method Detail

      • calculateProbability

        double calculateProbability​(String... tokens)
        Calculate the probability of a series of tokens (e.g. a sentence), given a vocabulary.
        Parameters:
        tokens - the text tokens to calculate the probability for.
        Returns:
        the probability of the given text tokens in the vocabulary
      • predictNextTokens

        @Deprecated
        StringList predictNextTokens​(StringList tokens)
        Predict the most probable output sequence of tokens, given an input sequence of tokens.
        Parameters:
        tokens - a sequence of tokens.
        Returns:
        the most probable subsequent token sequence
      • predictNextTokens

        String[] predictNextTokens​(String... tokens)
        Predict the most probable output sequence of tokens, given an input sequence of tokens.
        Parameters:
        tokens - a sequence of tokens
        Returns:
        the most probable subsequent token sequence