Package opennlp.tools.ml.maxent.quasinewton

Class QNMinimizer

java.lang.Object

opennlp.tools.ml.maxent.quasinewton.QNMinimizer

public class QNMinimizer
extends Object

Implementation of the Limited memory Broyden-Fletcher-Goldfarb-Shanno algorithm (L-BFGS) which supports L1-, L2-regularization and Elastic Net for solving convex optimization problems.

Usage example:

  // Quadratic function f(x) = (x-1)^2 + 10
  // f obtains its minimum value 10 at x = 1
  Function f = new Function() {

    @Override
    public int getDimension() {
      return 1;
    }

    @Override
    public double valueAt(double[] x) {
      return StrictMath.pow(x[0]-1, 2) + 10;
    }

    @Override
    public double[] gradientAt(double[] x) {
      return new double[] { 2*(x[0]-1) };
    }

  };

  QNMinimizer minimizer = new QNMinimizer();
  double[] x = minimizer.minimize(f);
  double min = f.valueAt(x);
 

See Also:

• Quasi-Newton Methods in: Nocedal J, Wright SJ. Numerical optimization. 2nd ed. New York: Springer; 2006.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	QNMinimizer.Evaluator	Evaluate the quality of training parameters.
static class	QNMinimizer.L2RegFunction	L2-regularized objective Function.

Field Summary

Fields

Modifier and Type	Field	Description
static final double	CONVERGE_TOLERANCE
static final double	INITIAL_STEP_SIZE	The initial step size: 1.0.
static final double	L1COST_DEFAULT	The default L1-cost value is 0.0d.
static final double	L2COST_DEFAULT	The default L2-cost value is 0.0d.
static final int	M_DEFAULT	The default number of Hessian updates to store is 15.
static final int	MAX_FCT_EVAL_DEFAULT	The default maximum number of function evaluations is 30,000.
static final double	MIN_STEP_SIZE	The minimum step size: 1e-10.
static final int	NUM_ITERATIONS_DEFAULT	By default the number of iterations is 100.
static final double	REL_GRAD_NORM_TOL

Constructor Summary

Constructors

Constructor	Description
QNMinimizer()	Initializes a QNMinimizer with default parameters (see: L1COST_DEFAULT and L2COST_DEFAULT).
QNMinimizer(double l1Cost, double l2Cost)	Initializes a QNMinimizer.
QNMinimizer(double l1Cost, double l2Cost, int iterations)	Initializes a QNMinimizer with L1 and L2 parameters.
QNMinimizer(double l1Cost, double l2Cost, int iterations, int m, int maxFctEval)	Initializes a QNMinimizer.

Method Summary

Modifier and Type	Method	Description
QNMinimizer.Evaluator	getEvaluator()
double[]	minimize(Function function)	Finds the parameters that minimize the objective function.
void	setEvaluator(QNMinimizer.Evaluator evaluator)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

CONVERGE_TOLERANCE

public static final double CONVERGE_TOLERANCE

See Also:

• Constant Field Values

REL_GRAD_NORM_TOL

public static final double REL_GRAD_NORM_TOL

See Also:

• Constant Field Values

INITIAL_STEP_SIZE

public static final double INITIAL_STEP_SIZE

The initial step size: 1.0.

See Also:

• Constant Field Values

MIN_STEP_SIZE

public static final double MIN_STEP_SIZE

The minimum step size: 1e-10.

See Also:

• Constant Field Values

L1COST_DEFAULT

public static final double L1COST_DEFAULT

The default L1-cost value is 0.0d.

See Also:

• Constant Field Values

L2COST_DEFAULT

public static final double L2COST_DEFAULT

The default L2-cost value is 0.0d.

See Also:

• Constant Field Values

NUM_ITERATIONS_DEFAULT

public static final int NUM_ITERATIONS_DEFAULT

By default the number of iterations is 100.

See Also:

• Constant Field Values

M_DEFAULT

public static final int M_DEFAULT

The default number of Hessian updates to store is 15.

See Also:

• Constant Field Values

MAX_FCT_EVAL_DEFAULT

public static final int MAX_FCT_EVAL_DEFAULT

The default maximum number of function evaluations is 30,000.

See Also:

• Constant Field Values

Constructor Details

QNMinimizer

public QNMinimizer()

Initializes a QNMinimizer with default parameters (see: L1COST_DEFAULT and L2COST_DEFAULT).

QNMinimizer

public QNMinimizer(double l1Cost,
 double l2Cost)

Initializes a QNMinimizer.

Parameters:

l1Cost - The L1-regularization cost. Must be equal to or greater than 0.

l2Cost - The L2-regularization cost. Must be equal to or greater than 0.

Throws:

IllegalArgumentException - Thrown if parameters were invalid.

QNMinimizer

public QNMinimizer(double l1Cost,
 double l2Cost,
 int iterations)

Initializes a QNMinimizer with L1 and L2 parameters.

Parameters:

l1Cost - The L1-regularization cost. Must be equal to or greater than 0.

l2Cost - The L2-regularization cost. Must be equal to or greater than 0.

iterations - The maximum number of iterations. Must be greater than 0.

Throws:

IllegalArgumentException - Thrown if parameters were invalid.

QNMinimizer

public QNMinimizer(double l1Cost,
 double l2Cost,
 int iterations,
 int m,
 int maxFctEval)

Initializes a QNMinimizer.

Parameters:

l1Cost - The L1-regularization cost. Must be equal to or greater than 0.

l2Cost - The L2-regularization cost. Must be equal to or greater than 0.

iterations - The maximum number of iterations. Must be greater than 0.

m - The number of Hessian updates to store. Must be greater than 0.

maxFctEval - The maximum number of function evaluations. Must be greater than 0.

Throws:

IllegalArgumentException - Thrown if parameters were invalid.

Method Details

getEvaluator

public QNMinimizer.Evaluator getEvaluator()

setEvaluator

public void setEvaluator(QNMinimizer.Evaluator evaluator)

minimize

public double[] minimize(Function function)

Finds the parameters that minimize the objective function.

Parameters:

function - The objective Function.

Returns:

The minimizing parameters.