Least Squares
HOME Math Software Downloads Numerical Methods Register Your Software Contact Search Credit

Software:
Linear Algebra
Functions
Ordinary Differential Equations
Systems of Nonlinear Equations
Multiple Integration
Maxima and Minima
Functions and Equations
Regression
Approximation & Interpolation
Stereographer
Math Miscellanea
Support Software
Numerical Methods:
Integration and Differentiation
Solution of Equations
Maxima and Minima
Approximation of Functions
Regression
Polynomial Regression
Fast Fourier Transforms
Differential Equations
Linear Algebra Methods
Miscellaneous Procedures
Decimal Comma/Decimal Point
                            

   
Google
 
REGRESSION

The method of LEAST SQUARES is used.

Linear Regression

To illustrate, if  z = A0 + A1*x + A2*y  is to be fitted to N data points, the normal equations are
 

A0*N    + A1*x     + A2*y      = z

A0*x + A1*x*x + A2*x*y = x*z

A0*y + A1*y*x + A2*y*y = y*z


where the
s are over all N data sets.

Solving the above system we get the values of the coefficients A0, A1 and A2. Obviously, it may be generalized to any number of variables or parameters.

The correlation coefficient is computed with

       [festimated - fmean]2
SQR ( )
           
[f-fmean]2

The standard error of estimate is calculated using

       [f - festimated ]2
SQR ( )
               N-2

Summations are over all N data points, f is the dependent variable, and the festimated is found using the regression results.

Curvilinear Regression

If we wish to fit the curve

F(W) = A0*G(x,y,z) + A1*H(x,y,z) + A2*J(x,y,z)

to N data points then the normal equations are

A0* G*G + A1* G*H + A2* G*J = G*F

A0* H*G + A1* H*H + A2* H*J = H*F

A0* J*G + A1* J*H + A2* J*J = J*F

where the summations are over all N data points.

Solving the above system we get the values of the coefficients A0, A1 and A2.

Obviously, it may be generalized to any number of variables or parameters.

The correlation coefficient and the standard error of estimate are calculated as in LINEAR REGRESSION.

General Regression

To illustrate the general method, let's say we want to fit a curve with three parameters, A, B and C, to N data points, that is, the curve

F(A,B,C,x)

where x standS for any number of variables.

If A0, B0 and C0 are sufficiently close approximations to A, B and C, then we can use Taylor's theorem to put

F(A,B,C,x) F(A0,B0,C0,x) + (A-A0)*Fa + (B-B0)*Fb + (C-C0)*Fc

where we ignored terms of order higher than the first and where

     F        F        F
Fa = , Fb = , Fc = , all evaluated at (
A0,B0,C0).
    
A        B        C

Now, let's call the residuals

R = F(A0,B0,C0,x) - F(A,B,C,x)

If we minimize the residuals using the method of least squares, we obtain the normal equations

(A-
A0)* Fa*Fa + (B-B0)* Fa*Fb + (C-C0)* Fa*Fc = - Fa*R

(A-A0)* Fb*Fa + (B-B0)* Fb*Fb + (C-C0)* Fb*Fc = - Fb*R

(A-A0)* Fc*Fa + (B-B0)* Fc*Fb + (C-C0)* Fc*Fc = - Fc*R

where Fa, Fb and Fc are as defined before and the summations are over all N data points.

If we solve this last set of linear equations for (A-A0), (B-B0) and (C-C0), we obtain corrections on the initial approximation, that is, if the solutions to the system are DA, DB and DC, where

A-A0=DA  ;  B-B0=DB  ;  C-C0=DC

then a closer approximation to the solution is

A1=A0+DA  ;  B1=B0+DB  ;  C1=C0+DC

We may now use A1, B1 and C1 to again set a system of linear equations and solve it to obtain A2, B2 and C2.  This is done until all parameters in the current approximation in the sequence differ from those of the previous approximation by less than their specified error and the change in R is less than its specified error.  Of course, the method may be generalized to any number of variables or parameters.

The correlation coefficient and the standard error of estimate are calculated as in LINEAR REGRESSION.

Copyright 2001-2010 Numerical Mathematics. All rights reserved.