Numerical Methods · Module 9 of 10
Numerical Differentiation
A derivative is a slope, and a slope can be estimated from nearby values. The centered difference cancels the leading error and is far more accurate than the one-sided forms, the lesson that underlies every ODE solver.
Readiness check
This module estimates derivatives from values. Tick only what you can do closed-notes.
- Recall a derivative as a limit of a difference quotient.
- Evaluate a function at x + h and x − h.
- Recall the order O(hn) of an error.
- Recall the Taylor series of a function.
- Subtract two close numbers and notice lost digits.
The core idea
A derivative is approximated by a finite difference of nearby function values. The one-sided forms are first-order accurate; the centered difference cancels the leading error term and is second-order, far better for the same step.
forward: (f(x+h) − f(x))/h, O(h)centered: (f(x+h) − f(x−h))/(2h), O(h2)Richardson: combine two h to gain orderDifferentiation reverses integration: instead of an area, it estimates a slope. From the Taylor series, the derivative at a point can be approximated by differences of nearby values. The forward difference uses the point and one ahead, (f(x + h) − f(x))/h; the backward difference uses one behind. Both carry a leading error proportional to h, so they are first-order accurate. The centered difference, (f(x + h) − f(x − h))/(2h), subtracts symmetric points so the first-order error terms cancel, leaving an error proportional to h2, a major improvement for the same spacing. The same Taylor expansions give formulas for the second derivative and for higher accuracy. Richardson extrapolation goes further, combining two centered estimates at h and h/2 to cancel the next error term and gain two more orders. The catch is roundoff: making h too small subtracts nearly equal numbers and loses precision, so an optimal step balances truncation against roundoff.
The skills, taught in order
Five skills build the difference formulas, their accuracy, and the extrapolation that sharpens them.
9.1 Finite-difference approximations
The Taylor series turns a derivative into a combination of function values at nearby points. Truncating it after the derivative of interest gives a finite-difference formula, with the first dropped term setting the order of accuracy.
9.2 Forward and backward differences
The forward difference (f(x + h) − f(x))/h and the backward difference (f(x) − f(x − h))/h each use the point and one neighbour. Both are first-order, O(h), so their error halves only when h halves, modest accuracy useful at the ends of a data set.
| Formula | Points used | Accuracy |
|---|---|---|
| Forward | x, x + h | O(h) |
| Backward | x − h, x | O(h) |
| Centered | x − h, x + h | O(h2) |
The centered difference, using points either side, is an order more accurate than the one-sided forms for the same step.
9.3 The centered difference
The centered difference (f(x + h) − f(x − h))/(2h) is second-order accurate because the symmetric Taylor terms cancel the O(h) error. It is the default first-derivative formula wherever points exist on both sides.
9.4 The order of accuracy
The order tells how fast the truncation error falls as h shrinks: halving h cuts an O(h) error in half but an O(h2) error to a quarter. Knowing the order lets you predict accuracy and compare formulas before computing.
9.5 Richardson extrapolation
Richardson extrapolation combines two centered estimates at h and h/2 to cancel the leading error term: D ≈ (4 D(h/2) − D(h))/3, gaining two orders of accuracy. The same idea, applied to integration, yields Romberg integration.
Engineering connection: finite differences turn governing equations into the algebraic systems solved in Computational Fluid Dynamics, and they are the step inside every ODE solver.
Worked example 1: forward, backward, and centered differences
Estimate the derivative of f(x) = ex at x = 0 (true value 1) using forward, backward, and centered differences with h = 0.1, and compare the errors.
- ProblemEstimate f′(0) for ex by three difference formulas, as in Figure 1.
- Given / findf(x) = ex, x = 0, h = 0.1, true f′ = 1. Find the three estimates and their errors.
- Assumptionsf is smooth; h is small enough that roundoff is negligible.
- ModelForward (f(h) − f(0))/h; backward (f(0) − f(−h))/h; centered (f(h) − f(−h))/(2h).
- Equationse0.1 = 1.105171, e−0.1 = 0.904837
- SolveForward = (1.105171 − 1)/0.1 = 1.05171 (5.17%). Backward = (1 − 0.904837)/0.1 = 0.95163 (4.84%). Centered = (1.105171 − 0.904837)/0.2 = 1.00167 (0.167%).
- CheckThe two one-sided errors are near 5 percent and lie on opposite sides of the true value; their average is close to the centered result, which is why symmetry cancels the leading error.
- ConclusionThe centered difference is about thirty times more accurate than the one-sided forms for the same h, the payoff of second-order accuracy.
Worked example 2: Richardson extrapolation
Sharpen the derivative of ex at x = 0 by Richardson extrapolation, combining centered estimates at h = 0.2 and h = 0.1.
- ProblemApply Richardson extrapolation to the centered estimates of f′(0), as in Figure 2.
- Given / findCentered estimates D(0.2) and D(0.1) for ex at 0, true f′ = 1. Find the extrapolated value.
- AssumptionsBoth inputs are centered (second-order), so the leading error is O(h2).
- ModelD ≈ (4 D(h/2) − D(h))/3, which cancels the h2 term and leaves O(h4).
- EquationsD(0.2) = (e0.2 − e−0.2)/0.4D = (4 D(0.1) − D(0.2))/3
- SolveD(0.2) = (1.221403 − 0.818731)/0.4 = 1.006680. D(0.1) = 1.001668 (from Worked Example 1). D = (4 × 1.001668 − 1.006680)/3 = (4.006672 − 1.006680)/3 = 2.999992/3 = 0.999997.
- CheckThe extrapolated error is about 3 parts per million, versus 0.17 percent for the better input, confirming the two-order gain from cancelling the h2 term.
- ConclusionRichardson extrapolation turns two modest estimates into a near-exact one for almost no extra work, the same idea that powers Romberg integration.
Misconceptions and diagnostics
| Mistake | Symptom | Diagnostic question | Correction |
|---|---|---|---|
| One-sided where centered fits | Needlessly large error | "Are there points on both sides?" | Use the centered difference when possible. |
| Making h too small | Error grows from roundoff | "Am I subtracting near-equal numbers?" | Balance truncation against roundoff with a moderate h. |
| Wrong denominator | Centered formula divided by h, not 2h | "Did I span 2h between the points?" | The centered difference divides by 2h. |
| Ignoring the order | Accuracy misjudged | "Is this O(h) or O(h2)?" | Use the order to predict how error scales with h. |
Practice ladder
For f(x) = x2, estimate f′(2) by the centered difference with h = 0.1.
Show answer
(f(2.1) − f(1.9))/0.2 = (4.41 − 3.61)/0.2 = 0.8/0.2 = 4.0, exactly f′(2) = 4 (centered is exact for quadratics).
For the same function, estimate f′(2) by the forward difference with h = 0.1 and find the error.
Show answer
(f(2.1) − f(2))/0.1 = (4.41 − 4)/0.1 = 4.1; error 0.1 (2.5 percent), worse than the exact centered result.
If a centered difference has 0.4 percent error at h = 0.2, estimate its error at h = 0.1.
Show answer
Centered error is O(h2), so halving h cuts it to about a quarter: roughly 0.1 percent.
You must estimate a velocity from noisy position-versus-time data. Explain why a smaller step is not always better here.
What good work looks like
Shrinking h reduces truncation error but amplifies the measurement noise, since the difference of two close noisy readings is dominated by noise; an optimal step balances the two, often with smoothing before differentiating.
Working with AI, and proving it yourself
Use AI as an examiner, not a solver
Portfolio task
Estimate a derivative from real data by forward, backward, and centered differences, compare to a known value, and sharpen it with Richardson extrapolation.
Retrieval and spaced review
Closed notes. Answer out loud, then reveal.
1. Write the centered difference.
(f(x + h) − f(x − h))/(2h), accurate to O(h2).
2. Why is the centered form more accurate?
Symmetric points cancel the leading first-order error term.
3. Give the orders of the one-sided forms.
Forward and backward differences are both O(h).
4. What does Richardson extrapolation do?
Combines two estimates to cancel the leading error and gain accuracy.
5. Why can h be too small?
Subtracting near-equal values loses precision, so roundoff grows.
Textbook mapping
This module follows Chapra and Canale, Numerical Methods for Engineers, 7th edition. Use these references to read further.
| Topic in this module | Where to read more |
|---|---|
| Finite-difference approximations | Chapra & Canale, Chapter 23 |
| High-accuracy and centered differences | Chapra & Canale, Chapter 23 |
| Richardson extrapolation | Chapra & Canale, Chapters 22 and 23 |
Chapter numbers refer to the 7th edition. The difference formulas are standard, so any recent edition will align closely.