isanet.optimizer.linesearch

isanet.optimizer.linesearch.line_search_wolfe(phi, derphi, phi0=None, old_phi0=None, derphi0=None, c1=0.0001, c2=0.9, amax=None, maxiter=10, verbose=0)

Return alpha > 0 that satisfies strong Wolfe conditions in order to get a descent direction or the last alpha found if the line search algorithm did not converge. For major details on the implementation refer to Wright and Nocedal, ‘Numerical Optimization’, 1999, pp. 59-61.

Parameters

  • phi (callable phi(alpha)) – Objective scalar function.

  • derphi (callable phi'(alpha)) – Objective function derivative. Returns a scalar.

  • phi0 (float, optional) – Value of phi at 0.

  • old_phi0 (float, optional) – Value of phi at previous point.

  • derphi0 (float, optional) – Value of derphi at 0

  • c1 (float, optional) – Parameter for Armijo condition rule.

  • c2 (float, optional) – Parameter for curvature condition rule.

  • amax (float, optional) – Maximum step size.

  • maxiter (int, optional) – Maximum number of iterations to perform.

Returns

alpha_star – Best alpha, or last alpha if the line search algorithm did not converge.

Return type

float

isanet.optimizer.linesearch.line_search_wolfe_f(phi, derphi, phi0=None, c1=0.0001, c2=0.9)

a fast strong Wolfe line search

class isanet.optimizer.linesearch.phi_function(model, optimizer, w, X, Y, d)

Bases: object

A wrapper for the phi function which provides phi() and derphi() methods used in the line search function.

Parameters

  • model (isanet.model.MLP) – Specify the Multilayer Perceptron object to the phi objcet

  • optimizer (isanet.optimizer.Optimizer) – Specify the optimizer that provides the evaluation function and the gradient evaluation

  • w (arrays-like of shape (n_variables, 1)) – column vector specifying the current point

  • X (array-like of shape (n_samples, n_features)) – The input data.

  • Y (array-like of shape (n_samples, n_output)) – The target values.

  • d (array-like of shape (n_variables, 1)) –

derphi(a)

Compute the dot producte between the gradient and the direction when an alpha ‘a’ parameter is passed:

phips = g_a^T*d
Parameters

a (scalar) – alpha parameter

phi(a)

Compute the phi value when an alpha ‘a’ parameter is passed

Parameters

a (scalar) – alpha parameter