AN ADAPTIVE NONLINEAR LEAST-SQUARES ALGORITHMS
Original Publication Date: 1979-Jul-31
Included in the Prior Art Database: 2007-Mar-29
Software Patent Institute
Dennis Jr., John E.: AUTHOR [+4]
AN ADAPTIVE ~~JONLXNEAR LEAST-SQUARES~ .-- /---, --'.* *Cornell University. Research supported in part by NSF Grant MCS76-00324
AN ADAPTIVE ~~JONLXNEAR LEAST-SQUARES~ .-- /---, --'.*
*Cornell University. Research supported in part by NSF Grant MCS76-00324
* *NBER Computer Research Center
Research supported in part by NSF Grant MCS76-00324
NSF Grant MCS78-09525
United States Army Contract
tMassachusetts Institute of
Technology. Research supported in part by NSF Grants SOC76-14311 and MC:S76-00324
John E. Dennis, Jr.*, David M. Gay**, and Roy E. Welscht
Revised July 1979
NL2SOL is a modular program for solving nonlinear least- squares problems that incorporates a number of novel features.
It maintains a secant approximation S to the second-order
part of the least-squares Hessian and adaptively decides when to use this approximation. S is "sized" before updating,
something which is similar t o Oren-Luenberger scaling. The step
choice algorithm is based on minimizing a local quadratic model
of the sum of squares function const:rained t o an elliptical trust region centered a t the current approximate minimizer. This is accomplished using ideas discussed by Xor6, together with a special module for assessing the quality of the step thus computed. These and other ideas behind ML2SOL stre discussed and its evolution and current implementation are also described briefly.
MI ADAPTIVE NOllLINEAR LEAST-SQUARES ALGORITHM
bv 2. The llonlincsr Least-Squaras Problem
There are good reasons for numerical analysts to study this
J.E. Dennis, Jr., David H. Gay, Roy E. Welsch problem. In the first place, it is a computation of prisary i=- portance i n statistical data analysis and hence in the social
This project began in order to meet a need a t the Computer Research Center of the llational Bureau of Economic Research for
a nonlinear least-squares algorithm which, in the large residual case, uould be more reliable than the Gauss-Newton or Levenberg-
P!arquardt nethod [Dennis, 19771 and more efficient than the secant or variable metric algorithms [Dennis 6 Mor6, 19771 such as the DaviCon-Fletcher-bkell, which 6-e inteded for g e n d function mirlrP;ization.
We have developed a satisfactory computer program called
XLZSOL based on ideas in [Dennis 5 Weisch, 19761 and o u ~ primary
parpose here i s to report the details and to give some test results. Ca the othsr hand, we learned so much during the development which
seess likely to be applicabiie in the development of other algoriths
that we have chosen to expand our exposition t o include tlds experiexe.
sciences, a8 well as in the more traditional areas within the physical sciences. Th...