Table 1 Optimization methods from minimize

From: SciPy 1.0: fundamental algorithms for scientific computing in Python

	Nelder-Mead	Powell	COBYLA	CG	BFGS	L-BFGS-G	SLSQP	TNC	Newton-CG	dogleg	trust-ncg	trust-exact	trust-Krylov
Version added	0.6*	0.6*	0.6*	0.6*	0.6*	0.6*	0.9	0.6*	0.6*	0.13	0.13	0.19	1.0
Wrapper			✓			✓	✓	✓					✓
First derivatives				✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Second derivatives					~	~	~	✓	✓	✓	✓	✓	✓
Iterative Hessian factorization								✓	✓		✓		✓
Local convergence				L	S	L	S	S*	S*	Q	S*	Q	S*
Global convergence					✓	✓	✓	✓	✓	✓	✓	✓	✓
Trust region	Neither	LS	TR	LS	LS	LS	LS	LS	LS	TR	TR	TR	TR
Bound constraints						✓	✓	✓	✓
Equality constraints							✓
Inequality constraints			✓				✓
References	^98,99	¹⁰⁰	^101,102,103	^104,105	¹⁰⁵	^106,107	^{108,109,110,111}	¹¹²	¹⁰⁵	^105,113	^105,114	^115,116	^62,117

Optimization methods from minimize, which solves problems of the form \({\mathrm{min}}_xf\left(x\right)\), where \(x \in {\Bbb R}^n\) and \(f:{\Bbb R}^n \to {\Bbb R}\). ‘Version added’ specifies the algorithm’s first appearance in SciPy. Algorithms with version added “0.6*” were added in version 0.6 or before. ‘Wrapper’ indicates whether the implementation available in SciPy wraps a function written in a compiled language (for example, C or FORTRAN). ‘First derivatives’ and ‘second derivatives’ indicate whether first or second order derivatives are required. When ‘second derivatives’ is flagged with ‘~’, the algorithm accepts but does not require second-order derivatives from the user; it computes an approximation internally and uses it to accelerate method convergence.
‘Iterative Hessian factorization’ denotes algorithms that factorize the Hessian in an iterative way, which does not require explicit matrix factorization or storage of the Hessian. ‘Local convergence’ gives a lower bound on the rate of convergence of the iteration sequence once the iterate is sufficiently close to the solution: linear (L), superlinear (S) and quadratic (Q). Convergence rates denoted S* indicate that the algorithm has a superlinear rate for the parameters used in SciPy, but can achieve a quadratic convergence rate with other parameter choices. ‘Global convergence’ is marked for the algorithms with guarantees of convergence to a stationary point (that is, a point x* for which \(\nabla f\left( {x^ \ast } \right) = 0\)); this is not a guarantee of convergence to a global minimum. ‘Lines-search’ (LS) or ‘trust-region’ (TR) indicates which of the two globalization approaches is used by the algorithm. The table also indicates which algorithms can deal with constraints on the variables. We distinguish among bound constraints (\(x^l \le x \le x^u\)), equality constraints (\(c_{{\mathrm{eq}}}\left( x \right) = 0\)) and inequality constraints (\(c_{{\mathrm{ineq}}}\left( x \right) \ge 0\)).

Back to article page

Table 1 Optimization methods from minimize

Search

Quick links