Home / Courses / VVUQ / Calibration and Prediction

VVUQ · Module 9 of 10

Model Calibration and Predictive Capability

Calibrating a model to data tunes its parameters to fit; it is not the same as validating it. The real goal is predictive capability: trustworthy prediction at conditions the model was not fitted to, with the uncertainty to match.

Readiness check

This module fits parameters and predicts. Tick only what you can do closed-notes.

Fit a parameter by least squares.
Combine independent uncertainties in quadrature.
Distinguish fitting data from testing against data.
Recall model-form uncertainty from earlier modules.
Recall that predicting outside a range is extrapolation.

0 or 1 weak itemsContinue with this module.

2 weak itemsRevisit least-squares calibration in Optimization, Module 5.

3 or more weak itemsRevisit validation uncertainty in Module 5.

The core idea

Calibration tunes model parameters to fit data; it improves agreement but does not prove the model. Predictive capability is trustworthy prediction beyond the fitted data, and its uncertainty combines the calibrated parameter uncertainty with the model-form uncertainty.

calibration: fit parameters to datavalidation: test against independent datau_pred = √(u_param² + u_model²)

Models almost always contain parameters that are not known from first principles, a friction coefficient, a heat-transfer constant, a material property. Calibration estimates these by fitting the model to data, typically by least squares, so the model reproduces the calibration set. This is valuable but carries a trap: a model tuned to data will match that data almost regardless of whether the underlying physics is right, so calibration is not validation. Validation must use independent data the model was not fitted to. The ultimate aim is predictive capability: the model's ability to predict, with quantified uncertainty, at conditions where no data were used to tune it. The uncertainty of such a prediction has two parts that combine in quadrature: the parameter uncertainty u_param left over from the calibration (the fitted values are not exact), and the model-form uncertainty u_model estimated from validation (the equations are approximate). A prediction reported without both is incomplete, and extrapolation beyond the calibration and validation range inflates both.

The skill works when: you calibrate on one set, validate on another, and combine both uncertainties in the prediction.

The skill breaks down when: calibration agreement is mistaken for validation, or model-form uncertainty is left out of the prediction.

The concept. Calibration fits the model to a data set; predictive capability is trustworthy prediction beyond it, carrying both the parameter and model-form uncertainty.

The skills, taught in order

Five skills build calibration, its distinction from validation, and the uncertainty of a prediction.

9.1 Model calibration

Calibration estimates unknown model parameters by fitting the model to data, usually minimising the sum of squared residuals as in nonlinear least squares. The result is a best-fit parameter value with an associated uncertainty from the scatter of the fit.

9.2 Calibration is not validation

A calibrated model matches its calibration data by construction, so good agreement there proves nothing about the physics. Validation requires independent data not used in the fit. Confusing the two, calibrating and validating on the same set, is a common and serious error.

Activity	Uses data to	Tells you
Calibration	tune parameters	best-fit parameters
Validation	test the tuned model	model-form error
Prediction	neither (new condition)	the answer, with uncertainty

Calibration and validation must use different data. Prediction applies the validated, calibrated model to a new condition.

9.3 Predictive capability

Predictive capability is the model's ability to predict at conditions where no tuning data exist, the real purpose of modeling. It rests on a calibrated model, validated on independent data, applied within or just beyond its validated domain.

9.4 Prediction uncertainty

A prediction's uncertainty combines the parameter uncertainty from calibration and the model-form uncertainty from validation, in quadrature: u_pred = √(u_param² + u_model²). Both must be included; omitting the model-form term understates the true uncertainty.

9.5 Extrapolation and its cost

Predicting beyond the calibration and validation range is extrapolation, where both parameter and model-form uncertainty grow and may be unquantifiable. The further from the data, the weaker the prediction, so extrapolation demands explicit justification and inflated uncertainty.

Engineering connection: a calibrated fatigue or heat-transfer model is only trustworthy for design once it is validated on separate data and its prediction carries both uncertainty terms.

Worked example 1: calibrating a parameter

A linear model F = k·x is calibrated to the data (1, 2.1), (2, 3.9), (3, 6.2) by least squares. Find the best-fit parameter k.

Figure 1. The best-fit line through the origin has slope k, the calibrated parameter. Least squares chooses k to minimise the squared residuals.

ProblemCalibrate the parameter k for the data in Figure 1.
Given / findF = k·x, data (1, 2.1), (2, 3.9), (3, 6.2). Find k.
AssumptionsThe model passes through the origin; errors are in F only.
ModelLeast squares for a proportional model: k = Σ(x_iF_i)/Σ(x_i²).
EquationsΣ(xF) = 1(2.1) + 2(3.9) + 3(6.2) = 28.5Σ(x²) = 1 + 4 + 9 = 14
Solvek = 28.5/14 = 2.036.
CheckThe fit gives F ≈ 2.036x, close to each data point (2.04, 4.07, 6.11 versus 2.1, 3.9, 6.2). The small residuals give a parameter uncertainty used in prediction.
ConclusionThe calibrated parameter is k = 2.036. This tunes the model to the data, but validating it requires a separate data set.

Result. Calibrated k = 2.036.

Worked example 2: the uncertainty of a prediction

Using the calibrated model to predict at a new condition, the calibration leaves a parameter uncertainty u_param = 0.03 (relative) and validation gives a model-form uncertainty u_model = 0.05. Find the prediction uncertainty.

Figure 2. The prediction uncertainty combines the leftover calibration (parameter) uncertainty and the validation (model-form) uncertainty in quadrature. Here the model-form term dominates.

ProblemFind the prediction uncertainty for the case in Figure 2.
Given / findu_param = 0.03, u_model = 0.05. Find u_pred.
AssumptionsThe two contributions are independent; the prediction is within the validated domain.
Modelu_pred = √(u_param² + u_model²).
Equationsu_pred = √(0.03² + 0.05²)
Solveu_pred = √(0.0009 + 0.0025) = √0.0034 = 0.058.
CheckThe model-form term (0.05) dominates the parameter term (0.03), so improving the model matters more than a tighter calibration here. Omitting u_model would understate the uncertainty by nearly half.
ConclusionThe prediction carries about a 5.8% uncertainty, combining calibration and model-form contributions. Both are essential to an honest prediction.

Result. u_pred = 0.058 (about 5.8%).

Misconceptions and diagnostics

Mistake	Symptom	Diagnostic question	Correction
Calibration counted as validation	Model tested on its fit data	"Was this data used to tune?"	Validate on independent data.
Omitting model-form uncertainty	Prediction uncertainty too small	"Did I include u_model?"	Combine parameter and model-form uncertainty.
Over-fitting parameters	Great fit, poor prediction	"Are there too many free parameters?"	Keep the model physically constrained.
Extrapolating a calibration	Confident prediction far from data	"Is this within the validated range?"	Inflate uncertainty and justify extrapolation.

Practice ladder

Level 1 · Direct skill

Calibrate F = k·x to (1, 3), (2, 5), (3, 8) by least squares.

Show answer

k = Σ(xF)/Σ(x²) = (3 + 10 + 24)/(1 + 4 + 9) = 37/14 = 2.64.

Level 2 · Mixed concept

Combine a parameter uncertainty of 0.04 with a model-form uncertainty of 0.03 for a prediction.

Show answer

u_pred = √(0.04² + 0.03²) = √0.0025 = 0.05.

Level 3 · Independent problem

A prediction reports only u_param = 0.03 and claims 0.03 total. What is missing and what is the honest value if u_model = 0.06?

Show answer

The model-form uncertainty is missing. Honest u_pred = √(0.03² + 0.06²) = √0.0045 = 0.067, more than double the reported value.

Transfer task | Real engineering

You calibrated a model on all your data and it fits well. Explain why you cannot yet trust its predictions and what to do.

What good work looks like

Fitting all the data leaves none for validation, so the good fit proves nothing about predictive capability. Reserve independent data (or gather more) to validate, estimate the model-form uncertainty, and report predictions with both parameter and model-form terms.

Working with AI, and proving it yourself

Use AI as an examiner, not a solver

"Check that my calibration and validation use different data."

"Give me three predictions; I will combine their two uncertainty terms."

"Calibrate the model for me." Fitting and validating yourself is the skill.

"Is the fit good enough?" Judging predictive capability is the point.

Portfolio task

Calibrate a model on one data set, validate it on another, and report a prediction at a new condition with both the parameter and model-form uncertainty combined.

Must include: a calibration, an independent validation, and a prediction with u_pred.

Retrieval and spaced review

Closed notes. Answer out loud, then reveal.

1. What does calibration do?

Estimates unknown model parameters by fitting the model to data.

2. Why is calibration not validation?

A tuned model matches its fit data regardless of the physics; validation needs independent data.

3. What is predictive capability?

Trustworthy prediction, with uncertainty, at conditions not used to tune the model.

4. Write the prediction uncertainty.

u_pred = √(u_param² + u_model²).

5. What happens under extrapolation?

Both uncertainties grow and may become unquantifiable.

TodayFinish this quiz and Levels 1 and 2 of the ladder.

+1 dayRe-derive the calibration and prediction uncertainty from a blank page.

+3 daysCalibrate and predict for three new models.

+7 daysTurn evidence into a decision, Module 10.

+30 daysReuse the calibration-validation split on a real model.

Standards mapping

This module follows the ASME Verification, Validation, and Uncertainty Quantification standards. Use these references to read further.

Topic in this module	Where to read more
Calibration versus validation	ASME V&V 10, Computational Solid Mechanics
Predictive capability and uncertainty	ASME VVUQ 10.2, UQ in Solid Mechanics
Model updating and prediction	Oberkampf and Roy, Verification and Validation in Scientific Computing

Standard designations refer to the ASME V&V series; the calibration-validation distinction is emphasised throughout the VVUQ literature.