Home / Courses / Numerical Methods / Regression

Numerical Methods · Module 6 of 10

Curve Fitting: Least-Squares Regression

Measured data scatter. Regression draws the single line that sits closest to all of it, in the least-squares sense, and reports how well that line explains the data through one number, r squared.

Readiness check

This module fits a line to data. Tick only what you can do closed-notes.

Compute a mean and a sum.
Recall the equation of a straight line y = a₀ + a₁x.
Square a residual and add the squares.
Take a logarithm to linearize a relation.
Distinguish a fit that interpolates from one that smooths.

0 or 1 weak itemsContinue with this module.

2 weak itemsRefresh statistics in Mathematics, Module 15.

3 or more weak itemsReview error measures in Module 1.

The core idea

Least-squares regression chooses the line that minimises the sum of squared vertical residuals. Its slope and intercept come from the normal equations, and the coefficient of determination measures how much of the scatter the line explains.

minimise S_r = Σ (y_i − a₀ − a₁x_i)²a₁ = (nΣxy − ΣxΣy)/(nΣx² − (Σx)²)r² = (S_t − S_r)/S_t

When data carry measurement noise, fitting a curve through every point is wrong; the goal is the trend, not the scatter. Least-squares regression finds the line y = a₀ + a₁x that minimises S_r, the sum of the squared vertical distances from the points to the line. Setting the derivatives of S_r with respect to the two coefficients to zero gives the normal equations, solved in closed form for the slope a₁ and intercept a₀. The quality of the fit is the coefficient of determination r² = (S_t − S_r)/S_t, the fraction of the total spread S_t that the line accounts for: r² = 1 is a perfect fit, r² = 0 explains nothing. The standard error of the estimate measures the typical residual. Many nonlinear laws, exponential or power, are fitted by first linearising them with logarithms.

The skill works when: you compute the sums, solve for slope and intercept, and report r².

The skill breaks down when: regression is used to interpolate exactly, or a nonlinear law is fitted without linearising.

The concept. Each point's vertical distance to the line is a residual. Least squares picks the slope and intercept that make the sum of those squared distances as small as possible.

The skills, taught in order

Five skills build the least-squares line, its quality measures, and the linearisation of nonlinear laws.

6.1 The least-squares criterion

Regression minimises the sum of squared residuals S_r = Σ(y_i − a₀ − a₁x_i)². Squaring penalises large misses and makes the problem smooth and uniquely solvable, which is why least squares, not least absolute error, is the standard.

6.2 The normal equations

Setting the partial derivatives of S_r to zero gives two linear normal equations whose solution is a₁ = (nΣxy − ΣxΣy)/(nΣx² − (Σx)²) and a₀ = ȳ − a₁x̄. Four sums, Σx, Σy, Σxy, Σx², fix the line.

Quantity	Meaning	Formula
S_t	total spread about the mean	Σ(y_i − ȳ)²
S_r	spread about the line	Σ(y_i − a₀ − a₁x_i)²
r²	fraction explained	(S_t − S_r)/S_t
s_y/x	standard error	√(S_r/(n − 2))

The fit and its quality. S_t is the spread with no model; S_r is what remains after the line.

6.3 The coefficient of determination

r² = (S_t − S_r)/S_t is the fraction of the original spread the line explains. Its square root r is the correlation coefficient. A high r² means the line captures most of the variation; a low one warns the model is poor.

6.4 The standard error of the estimate

The standard error s_y/x = √(S_r/(n − 2)) is the typical size of a residual, in the units of y. It quantifies the scatter about the line and feeds prediction intervals, the spread expected around a fitted value.

6.5 Linearising nonlinear models

An exponential y = αe^βx becomes linear as ln y = ln α + βx; a power law y = αx^β becomes linear in log-log. Fitting the transformed data with the same normal equations recovers the nonlinear coefficients, extending regression far beyond straight lines.

Engineering connection: calibrating a sensor, fitting a material law, or reducing test data all rest on least-squares regression and its r².

Worked example 1: a least-squares straight line

Fit a least-squares line to the data x = 1, 2, 3, 4, 5, 6, 7 and y = 0.5, 2.5, 2, 4, 3.5, 6, 5.5. Find the slope and intercept.

Figure 1. The fitted line passes through the centroid of the data with a slope near 0.84. It does not touch every point; it minimises the total squared miss.

ProblemFind the least-squares slope and intercept for the data in Figure 1.
Given / findSeven (x, y) pairs. Find a₁ and a₀.
AssumptionsErrors are in y only; a straight line is an appropriate model.
Modela₁ = (nΣxy − ΣxΣy)/(nΣx² − (Σx)²); a₀ = ȳ − a₁x̄.
Equationsn = 7, Σx = 28, Σy = 24, Σxy = 119.5, Σx² = 140
Solvea₁ = (7·119.5 − 28·24)/(7·140 − 28²) = (836.5 − 672)/(980 − 784) = 164.5/196 = 0.8393. a₀ = 24/7 − 0.8393·(28/7) = 3.4286 − 3.3571 = 0.0714.
CheckThe line passes through the centroid (x̄, ȳ) = (4, 3.4286): 0.0714 + 0.8393·4 = 3.4286 ✓, as every least-squares line must.
ConclusionThe best-fit line is y = 0.0714 + 0.8393x. It captures the upward trend without chasing the scatter.

Result. y = 0.0714 + 0.8393x.

Worked example 2: the quality of the fit

For the line of Worked Example 1, the total spread is S_t = 22.714 and the spread about the line is S_r = 2.991. Find the coefficient of determination, the correlation coefficient, and the standard error.

Figure 2. The line removes about 87 percent of the original spread, leaving a small residual sum. The coefficient of determination quantifies that improvement in one number.

ProblemFind r², r, and s_y/x for the fit in Figure 2.
Given / findS_t = 22.714, S_r = 2.991, n = 7. Find r², r, s_y/x.
AssumptionsThe same straight-line model from Worked Example 1.
Modelr² = (S_t − S_r)/S_t; r = √(r²); s_y/x = √(S_r/(n − 2)).
Equationsr² = (S_t − S_r)/S_ts_y/x = √(S_r/(n − 2))
Solver² = (22.714 − 2.991)/22.714 = 19.723/22.714 = 0.868. r = √0.868 = 0.932. s_y/x = √(2.991/5) = √0.598 = 0.773.
Checkr² = 0.868 means the line explains about 87 percent of the spread, a good but not perfect fit, consistent with visible scatter about the line.
ConclusionThe fit explains 87 percent of the variation with a typical residual of 0.77. These numbers, not the line alone, report how trustworthy the model is.

Result. r² = 0.868, r = 0.932, s_y/x = 0.773.

Misconceptions and diagnostics

Mistake	Symptom	Diagnostic question	Correction
Forcing the line through points	A wiggly fit chasing noise	"Am I smoothing or interpolating?"	Regression smooths; it should not pass through every point.
High r² means correct model	A good r² on the wrong curve	"Do the residuals show a pattern?"	Check residuals; r² alone does not prove the model.
Fitting a nonlinear law directly	Poor fit to exponential data	"Should I linearise first?"	Transform with logarithms, then fit a line.
Standard error confused with slope error	Misreported uncertainty	"Is this the residual scatter?"	s_y/x is the spread of residuals, not the slope's error.

Practice ladder

Level 1 · Direct skill

For points (1, 1), (2, 3), (3, 5), find Σx, Σy, Σxy, Σx².

Show answer

Σx = 6, Σy = 9, Σxy = 1 + 6 + 15 = 22, Σx² = 1 + 4 + 9 = 14.

Level 2 · Mixed concept

Use those sums to find the least-squares slope (n = 3).

Show answer

a₁ = (3·22 − 6·9)/(3·14 − 36) = (66 − 54)/(42 − 36) = 12/6 = 2. The data lie exactly on a line of slope 2.

Level 3 · Independent problem

A fit has S_t = 50 and S_r = 8. Find r² and say how good the fit is.

Show answer

r² = (50 − 8)/50 = 42/50 = 0.84. The line explains 84 percent of the spread, a fairly good fit.

Transfer task | Real engineering

You calibrate a sensor and get r² = 0.999 but the residuals curve systematically. Explain what this tells you and what to do.

What good work looks like

A high r² with patterned residuals signals the wrong model shape: the linear fit is close but the true relation is curved. Fit a nonlinear form (or linearise it) so the residuals become random, not r² alone.

Working with AI, and proving it yourself

Use AI as an examiner, not a solver

"Check that my line passes through the data centroid."

"Give me three data sets; I will judge whether a line is the right model."

"Fit the line for me." Computing the sums and coefficients is the skill.

"Is r² high enough?" Reading the residual pattern is the point.

Portfolio task

Fit a least-squares line to real measured data, report r² and the standard error, and inspect the residuals for a pattern that would call for a nonlinear model.

Must include: the fitted coefficients, r², the standard error, and a residual check.

Retrieval and spaced review

Closed notes. Answer out loud, then reveal.

1. What does least squares minimise?

The sum of squared vertical residuals, S_r.

2. Write the slope from the normal equations.

a₁ = (nΣxy − ΣxΣy)/(nΣx² − (Σx)²).

3. Define r².

(S_t − S_r)/S_t, the fraction of spread the line explains.

4. What is the standard error of the estimate?

s_y/x = √(S_r/(n − 2)), the typical residual size.

5. How is an exponential law fitted?

Linearise it with logarithms, then apply the normal equations.

TodayFinish this quiz and Levels 1 and 2 of the ladder.

+1 dayRe-derive the line and r² from a blank page.

+3 daysFit three new data sets and report r².

+7 daysContrast smoothing with exact interpolation, Module 7.

+30 daysReuse regression to calibrate a real sensor.

Textbook mapping

This module follows Chapra and Canale, Numerical Methods for Engineers, 7th edition. Use these references to read further.

Topic in this module	Where to read more
Least-squares regression	Chapra & Canale, Chapter 17
Coefficient of determination and standard error	Chapra & Canale, Chapter 17
Linearisation of nonlinear models	Chapra & Canale, Chapter 17

Chapter numbers refer to the 7th edition. The regression formulas are standard, so any recent edition will align closely.