The method of LEAST SQUARES is used.

Linear Regression

To illustrate, if  z = A₀ + A₁*x + A₂*y  is to be fitted to N data points, the normal equations are
 

A₀*N    + A₁*åx     + A₂*åy      = åz

A₀*åx + A₁*åx*x + A₂*åx*y = åx*z

A₀*åy + A₁*åy*x + A₂*åy*y = åy*z

where the å’s are over all N data sets.

Solving the above system we get the values of the coefficients A₀, A₁ and A₂. Obviously, it may be generalized to any number of variables or parameters.

The correlation coefficient is computed with

       å [f_estimated - f_mean]²
SQR ( —————————————————————— )
            å [f-f_mean]²

The standard error of estimate is calculated using

       å [f - f_estimated ]²
SQR ( ————————————————————— )
               N-2

Summations are over all N data points, f is the dependent variable, and the f_estimated is found using the regression results.

Curvilinear Regression

If we wish to fit the curve

F(W) = A₀*G(x,y,z) + A₁*H(x,y,z) + A₂*J(x,y,z)

to N data points then the normal equations are

A₀*å G*G + A₁*å G*H + A₂*å G*J = å G*F

A₀*å H*G + A₁*å H*H + A₂*å H*J = å H*F

A₀*å J*G + A₁*å J*H + A₂*å J*J = å J*F

where the summations are over all N data points.

Solving the above system we get the values of the coefficients A₀, A₁ and A₂.

Obviously, it may be generalized to any number of variables or parameters.

The correlation coefficient and the standard error of estimate are calculated as in LINEAR REGRESSION.

General Regression

To illustrate the general method, let's say we want to fit a curve with three parameters, A, B and C, to N data points, that is, the curve

F(A,B,C,x)

where x standS for any number of variables.

If A₀, B₀ and C₀ are sufficiently close approximations to A, B and C, then we can use Taylor's theorem to put

F(A,B,C,x) ÷ F(A₀,B₀,C₀,x) + (A-A₀)*Fa + (B-B₀)*Fb + (C-C₀)*Fc

where we ignored terms of order higher than the first and where

     ¶F        ¶F        ¶F
Fa = —— , Fb = —— , Fc = —— , all evaluated at (A₀,B₀,C₀).
     ¶A        ¶B        ¶C

Now, let's call the residuals

R = F(A₀,B₀,C₀,x) - F(A,B,C,x)

If we minimize the residuals using the method of least squares, we obtain the normal equations

(A-A₀)*å Fa*Fa + (B-B₀)*å Fa*Fb + (C-C₀)*å Fa*Fc = - å Fa*R

(A-A₀)*å Fb*Fa + (B-B₀)*å Fb*Fb + (C-C₀)*å Fb*Fc = - å Fb*R

(A-A₀)*å Fc*Fa + (B-B₀)*å Fc*Fb + (C-C₀)*å Fc*Fc = - å Fc*R

where Fa, Fb and Fc are as defined before and the summations are over all N data points.

If we solve this last set of linear equations for (A-A₀), (B-B₀) and (C-C₀), we obtain corrections on the initial approximation, that is, if the solutions to the system are DA, DB and DC, where

A-A₀=DA  ;  B-B₀=DB  ;  C-C₀=DC

then a closer approximation to the solution is

A₁=A₀+DA  ;  B₁=B₀+DB  ;  C₁=C₀+DC

We may now use A₁, B₁ and C₁ to again set a system of linear equations and solve it to obtain A₂, B₂ and C₂.  This is done until all parameters in the current approximation in the sequence differ from those of the previous approximation by less than their specified error and the change in R² is less than its specified error.  Of course, the method may be generalized to any number of variables or parameters.

The correlation coefficient and the standard error of estimate are calculated as in LINEAR REGRESSION.