Global Convergence Properties of Conjugate Gradient Methods for Optimization
SIAM Journal on OptimizationPublished 1 February 1992
Jean Charles Gilbert, Jorge Nocedal
Citations1,031
SJR quartileQ1
SJR score1.39
SNIP1.79
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper explores the convergence of nonlinear conjugate gradient methods without restarts, and with practical line searches, covering two classes of methods that are globally convergent on smooth, nonconvex functions.
Abstract
This paper explores the convergence of nonlinear conjugate gradient methods without restarts, and with practical line searches. The analysis covers two classes of methods that are globally convergent on smooth, nonconvex functions. Some properties of the Fletcher–Reeves method play an important role in the first family, whereas the second family shares an important property with the Polak–Ribière method. Numerical experiments are presented.
Keywords
Computer ScienceMathematics
Practical Methods of Optimization
6,403 Citations2000R. Ian Fletcher
Mathematical ProgrammingRestart procedures for the conjugate gradient method
1,700 Citations1977M. J. D. Powell
The main purpose of this paper is to provide an algorithm with a restart procedure that takes account of the objective function automatically and to study a multiplying factor that occurs in the definition of the search direction of each iteration.
ACM Transactions on Mathematical SoftwareTesting Unconstrained Optimization Software
1,520 Citations1981Jorge J. Morè, B. S. Garbow +1 more
A relatwely large but easy-to-use collection of test functions and designed gmdelines for testing the reliability and robustness of unconstrained optimization software.
Mathematical ProgrammingSome numerical experiments with variable-storage quasi-Newton algorithms
809 Citations1989Jean Charles Gilbert, Claude Lemaréchal
It is found appropriate to use a diagonal matrix, generated by an update of the identity matrix, so as to fit the Rayleigh ellipsoid of the local Hessian in the direction of the change in the gradient.
SIAM ReviewConvergence Conditions for Ascent Methods. II: Some Corrections
499 Citations1971Philip Wolfe
IMA Journal of Numerical AnalysisDescent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line Search
493 Citations1985M. Al-Baali
Lecture notes in mathematicsNonconvex minimization calculations and the conjugate gradient method
384 Citations1984M. J. D. Powell
This work considers the global convergence of conjugate gradient methods without restarts, assuming exact arithmetic and exact line searches, when the objective function is twice continuously differentiable and has bounded level sets.
Journal of Optimization Theory and ApplicationsEfficient hybrid conjugate gradient techniques
259 Citations1990Djamal Touati-Ahmed, C. Storey
New criteria for restarting conjugate gradient algorithms that prove to be computationally very efficient are given and these criteria provide a descent property and global convergence for any conjugates gradient algorithm using a nonnegative update β.
ACM Transactions on Mathematical SoftwareRemark on “Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4]”
246 Citations1980David F. Shanno, K. H. Phua
The subroutine incorporates two nonlinear optimization methods, a conjugate gradient algorithm and a variable metric algorithm, with the choice of method left to the user, with two differences.
SIAM ReviewConvergence Properties of Algorithms for Nonlinear Optimization
200 Citations1986M. J. D. Powell
This paper reviews some of the most successful methods for unconstrained, constrained and nondifferentiable optimization calculations and suggests that practical considerations provide the main new ideas, and that subsequent theoretical studies give improvements to algorithms, coherence to the subject, and better understanding.
Mathematical ProgrammingQN-like variable storage conjugate gradients
115 Citations1983A. Buckley, A. Lenir
A new combined CG-QN algorithm which can use whatever storage is available and is presented to demonstrate that the new algorithm is never worse than CONMIN and that it is almost always better if even a small amount of extra storage is provided.
Mathematical ProgrammingSome convergence properties of the conjugate gradient method
107 Citations1976M. J. D. Powell
It has been conjectured that the conjugate gradient method for minimizing functions of several variables has a superlinear rate of convergence, but Crowder and Wolfe show by example that the conjecture is false.
SIAM Journal on Numerical AnalysisRate of Convergence of Several Conjugate Gradient Algorithms
88 Citations1972Arthur I. Cohen
It is shown in a neighborhood of the minimum that the error, when starting from a point of reinitialization, decreases by order 2 after n steps.
IBM Journal of Research and DevelopmentLinear Convergence of the Conjugate Gradient Method
72 Citations1972Harlan Crowder, Philip Wolfe
Numerische MathematikOn the relation between quadratic termination and convergence properties of minimization algorithms
38 Citations1977Josef Stoer
General conditions are given, which together with the quadratic termination property, will ensure that the algorithm locally converges at leastn-step quadratically to a local minimum for sufficiently smooth nonquadratic functionsf.
Numerische MathematikOn the relation between quadratic termination and convergence properties of minimization algorithms
21 Citations1977Peter Baptist, Josef Stoer