Logo   ZunZunHelp.com
for
ZunZun.com Online Curve Fitting
and Surface Fitting Web Site
  Logo

Go to ZunZun.com

User Support Home

 

Home
Using ZunZun
F.A.Q.
ZunZun News

 

Basics

About ZunZun.com

Linear vs. Non-linear Regression

Multiple Linear Regression

Absolute vs. Relative Error

 

Jump Start

 

About ZunZun.com

Here you can curve and surface fit your 2D and 3D data online with a rich set of error histograms, error plots, curve plots, surface plots, contour plots, VRML, and source code.

The author, James Phillips began to curve and surface fit when he was in R&D about 15 or 16 years ago. These were mostly X-ray transmission and backscatter curve and surface data sets from the measurement of steel and aluminum thickness. He also did work with X-ray fluorescence of zinc coatings on steel. The software tools he had on hand at the time were too expensive for general use. Since he was learning to program computers, he started writing curve fitting software, first in C and then in C++. He left Tokyo, Japan in early 2000, returning to Birmingham, Alabama, and started programming in Python.
 

Linear vs. Non-linear Regression

"Linear regression" is used for equations that are linear *in the coefficients*. Here are some specific examples:

y = mx + b

y = aX^0 + bX^1 + cX^2 + dX^3

z = a + bX + cY + dXY

These equations are all of the general form:

result = coefficient_1 * function_1 + coefficient_2 * function_2 + coefficient_3 * function_3 + ...

where function_1 can be x^2, function_2 can be sin(x), etc., etc.


Here is an example equation that is NOT linear in the coefficients;

y = exp(a*x)

where we want to find the coefficient "a." The format of this equation cannot be used for linear regression, because it is not in the general form just discussed.

This is where "non-linear regression" comes into play. The basic idea is:

1) Make an initial guess for the value of "a"
2) Using our data set, calculate the sum of squared error (SSQ)
3) Try to change a so as to reduce the SSQ calculated in step 2 above (sometimes using derivatives)
4) Repeat steps 1 through 3 until the SSQ is as low as we can make it

Note that step 4 means repeated iteration, so non-linear regression is often referred to as "using iterative methods,"  which means it is computationally more expensive than linear regression and with large data sets can take a long time.



Multiple Linear Regression

Suppose you have 2-dimensional XY data, and want to fit a straight line to this data. The equation is commonly written as:

y = mx + b
 

This can be rewritten in polynomial form as

y = ax^0 + bx^1
 

A quadratic is then

y = ax^0 + bx^1 + cx^2
 

and a cubic is then

y = ax^0 + bx^1 + cx^2 + dx^3
 

and so on.


Straight lines are done, let's move on. For multiple linear regression, the key idea is the data formatting for the regression.

Let's say you want to fit your X and Y 2D data to this equation:

y = ax^3 + bsin(x)
 

Format your data as columns of

x^3 sin(x)

and again regress against Y.

 

Now for n-dimensional regression. Let's do a simple XYZ 3D fit first. Start with the equation

z = ax + by
 

Format your data as columns of

X Y

and regress against Z. All done.

 

For a more complex 3D problem, say

Z = a + bcos(X*Y) + cexp(X/Y)

format your data as columns of

1 cos(X*Y) exp(X/Y)

and regress against Z.

 


Absolute vs. Relative Error

Start with absolute error, since it is simple.

absolute error = predicted value - actual value
 

So if your curve fit predicts an output of 11.0 when the actual value is 10.0, then

absolute error = predicted - actual = 11.0 - 10.0 = 1.0
 

So in this case absolute error is 1.0.

 

Relative error is

absolute error / actual
 

So using the above example,

relative error = absolute error / actual = 1.0 / 10.0 = 0.1
 

Multiply by 100 to get percent error.

percent error = relative error * 100 = 0.1 * 100 = 10%
 

In this example, the curve fit is off by 10 percent.

 


Home | Using ZunZun | F.A.Q. | ZunZun News

Questions or problems regarding this web site should be directed to kristi.weldon@gmail.com.
Copyright 2008-9 ZunZun.com is open source.
Last modified: 01/28/09.