Distinguish code verification, solution verification, validation, calibration, and uncertainty quantification.
Estimate discretization and input uncertainty for simple models.
Interpret validation evidence without equating calibration with prediction.
Build a context-of-use and risk-informed credibility argument.
Evidence standard
Complete all four lessons, reproduce the worked checks, run the lab, and correct the weekly quiz. Treat AI output as candidate evidence until independently verified.
A single percent agreement number cannot tell whether discrepancy came from code, discretization, inputs, experiment, or model form.
Learning objectives
Define and distinguish Code verification and Solution verification.
Apply the lesson method to the worked the vvuq taxonomy and error-source map case.
Evaluate evidence, uncertainty, and AI-assisted output before making a claim.
Readiness check
Before continuing, explain what decision this topic supports and name one upstream source that must be controlled.
Check your response
A sound answer names a specific engineering decision, its configuration, and a controlled requirement, model, dataset, interface, or standard that constrains the work.
Core idea
Verification asks whether the mathematical model is solved correctly. Validation asks whether the model adequately represents reality for its intended use. UQ characterizes how uncertainty in inputs, models, numerics, and measurements affects quantities of interest and decisions.
Key concepts
Code verification
Evidence that software correctly implements the intended mathematical algorithms.
Solution verification
Estimation of numerical error for a specific simulation solution.
Validation
Assessment of model adequacy through comparison with relevant experimental reality.
Uncertainty quantification
Characterization and propagation of uncertainty to model outputs and decisions.
Step-by-step explanation
Define the quantity of interest and intended decision.
Verify code with exact, manufactured, or benchmark solutions where appropriate.
Verify the specific solution using convergence and numerical-error evidence.
Compare with relevant experiments and propagate uncertainty before making a credibility claim.
Worked example
A CFD pressure drop is 4.8 kPa and a test reports 5.1 kPa. The mesh study estimates 0.15 kPa numerical uncertainty and the experiment reports 0.20 kPa standard uncertainty.
1
Comparison error is 4.8 - 5.1 = -0.3 kPa, or -5.88% relative to the test.
2
Keep numerical and experimental uncertainty separate before combining under declared assumptions.
3
Do not call the remaining gap model-form error exactly; uncertain inputs and boundary conditions also contribute.
4
Check code and solution verification before interpreting physical-model adequacy.
Result. The -0.3 kPa difference is an observed comparison, not a complete diagnosis. Credibility requires an uncertainty-aware decomposition and relevance to use.
Independent check. Each discrepancy source is named and no component is assigned more certainty than evidence supports.
Common misconceptions
Misconception
Correction
Validation means the model is true
Validation provides evidence of adequacy for defined conditions and quantities; it does not prove universal truth.
A tool output closes the question
A result remains a candidate until its inputs, method, configuration, uncertainty, and relevance have been checked.
Diagnostic questions
Where does model-form uncertainty enter?
Through imperfect governing assumptions, closures, constitutive relations, geometry idealization, and omitted physics.
What would make this work reproducible?
Controlled inputs, method or code, versions, assumptions, outputs, and a stated interpretation tied to the decision.
Practice ladder
Basic
Classify ten VVUQ activities into code verification, solution verification, validation, calibration, or UQ.
Intermediate
Build an error-source map for a finite-element stress result.
Advanced
Explain why agreement can be accidental when two errors cancel.
AI-assisted engineering task
Ask AI to classify an evidence list using definitions, then identify statements that overclaim certainty.
How to prove the AI output yourself
Check each label against the VVUQ taxonomy.
Inspect actual procedures and quantities of interest.
Require a technical reviewer to resolve ambiguous activities.
Retrieval and spaced review
Answer closed-notes today, then again after 1, 3, 7, and 30 days.
Define Code verification.
Evidence that software correctly implements the intended mathematical algorithms.
What role does Solution verification play here?
Estimation of numerical error for a specific simulation solution.
What must a reviewer be able to reconstruct?
Each discrepancy source is named and no component is assigned more certainty than evidence supports.
End-of-lesson summary
Verification asks whether the mathematical model is solved correctly. Validation asks whether the model adequately represents reality for its intended use. UQ characterizes how uncertainty in inputs, models, numerics, and measurements affects quantities of interest and decisions.
Student notes
Draw one error-source tree per simulation before reporting an agreement percentage.
Insist on nouns: code, equations, solution, experiment, quantity, decision. This prevents slogan-level V&V definitions.
7.2
Code and solution verification through beam deflection
Why this lesson matters
A plausible result can hide coding defects, inadequate discretization, or an equation applied outside its assumptions.
Learning objectives
Define and distinguish Manufactured solution and Discretization error.
Apply the lesson method to the worked code and solution verification through beam deflection case.
Evaluate evidence, uncertainty, and AI-assisted output before making a claim.
Readiness check
Before continuing, explain what decision this topic supports and name one upstream source that must be controlled.
Check your response
A sound answer names a specific engineering decision, its configuration, and a controlled requirement, model, dataset, interface, or standard that constrains the work.
Core idea
Code verification tests implementation against known mathematical truth. Solution verification estimates remaining numerical error in a particular calculation. Analytical checks and convergence studies are complementary, not substitutes for one another.
Key concepts
Manufactured solution
A constructed exact solution used to test code implementation and observed order.
Discretization error
Difference introduced by representing continuous equations on finite spatial or temporal resolution.
Observed order
The convergence rate inferred from a systematic refinement sequence.
Asymptotic range
Refinement regime where the theoretical error model approximately governs behavior.
Step-by-step explanation
Reproduce a known analytical or manufactured solution to challenge implementation.
Refine mesh or time step systematically with fixed physics and solver tolerances.
Check monotonicity, observed order, and whether results enter an asymptotic range.
Estimate numerical uncertainty using a stated method and assumptions.
Keep model applicability separate from numerical accuracy.
Worked example
For the beam in Module 4, analytical deflection is 2.061 mm. Three finite-element meshes predict 1.93, 2.02, and 2.05 mm.
1
Compute absolute errors relative to the analytical solution: 0.131, 0.041, and 0.011 mm.
2
The sequence approaches the reference and error decreases, supporting convergence.
3
Do not infer exact order without refinement ratios and element formulation.
4
Check that the analytical and FEA models share simply supported boundaries, center load, linear elasticity, and Euler-Bernoulli applicability.
Result. The finest result differs by -0.011 mm, about -0.53%. This is useful verification evidence under aligned mathematical assumptions, not validation of beam reality.
Independent check. Reference solution, refinement definition, solver tolerance, quantity extraction, and assumptions are documented and reproducible.
Common misconceptions
Misconception
Correction
Mesh convergence validates the model
It addresses numerical solution behavior, not physical adequacy against reality.
A tool output closes the question
A result remains a candidate until its inputs, method, configuration, uncertainty, and relevance have been checked.
Diagnostic questions
What must remain fixed in a mesh study?
Physics, geometry, boundary conditions, solver settings, extraction definition, and convergence tolerances except the intended refinement.
What would make this work reproducible?
Controlled inputs, method or code, versions, assumptions, outputs, and a stated interpretation tied to the decision.
Practice ladder
Basic
Compute the three absolute and relative errors.
Intermediate
Design a mesh-refinement table with element size, degrees of freedom, result, iteration error, and estimated discretization error.
Advanced
Explain how shear deformation could make both a converged FEA solution and Euler-Bernoulli reference inadequate for a deep beam.
AI-assisted engineering task
Ask AI to check refinement-table arithmetic and identify missing columns, without inferring an observed order from incomplete data.
How to prove the AI output yourself
Recompute errors.
Plot result versus resolution.
Check solver and iteration convergence.
Compare assumptions with the reference solution.
Retrieval and spaced review
Answer closed-notes today, then again after 1, 3, 7, and 30 days.
Define Manufactured solution.
A constructed exact solution used to test code implementation and observed order.
What role does Discretization error play here?
Difference introduced by representing continuous equations on finite spatial or temporal resolution.
What must a reviewer be able to reconstruct?
Reference solution, refinement definition, solver tolerance, quantity extraction, and assumptions are documented and reproducible.
End-of-lesson summary
Code verification tests implementation against known mathematical truth. Solution verification estimates remaining numerical error in a particular calculation. Analytical checks and convergence studies are complementary, not substitutes for one another.
Student notes
Keep code verification evidence, solution verification evidence, and model-validation evidence in separate notebook sections.
The arithmetic is intentionally simple. Spend time on what the comparison does and does not establish.
7.3
Validation, calibration, and uncertainty sources
Why this lesson matters
A model tuned to one dataset can reproduce it while making poor predictions elsewhere. Calibration and validation must remain distinguishable.
Learning objectives
Define and distinguish Calibration and Validation experiment.
Apply the lesson method to the worked validation, calibration, and uncertainty sources case.
Evaluate evidence, uncertainty, and AI-assisted output before making a claim.
Readiness check
Before continuing, explain what decision this topic supports and name one upstream source that must be controlled.
Check your response
A sound answer names a specific engineering decision, its configuration, and a controlled requirement, model, dataset, interface, or standard that constrains the work.
Core idea
Calibration estimates model parameters using data under stated assumptions. Validation evaluates predictive adequacy using relevant, preferably independent data. Aleatory variability and epistemic uncertainty require different interpretations even when both are represented probabilistically.
Key concepts
Calibration
Inference of parameter values or distributions from observations and a model.
Validation experiment
An experiment designed to measure quantities and conditions relevant to assessing model adequacy.
Aleatory uncertainty
Variability represented as irreducible within the chosen model and information state.
Epistemic uncertainty
Lack of knowledge that may be reduced or better characterized with information.
Step-by-step explanation
Define the prediction question and validation hierarchy.
Design experiments with controlled inputs, measured boundary conditions, and uncertainty estimates.
Separate data used for parameter calibration from data used for predictive assessment.
Compare quantities using metrics that preserve sign, scale, uncertainty, and physical meaning.
Report the validated domain and prediction limitations rather than a universal validity label.
Worked example
A convection coefficient is calibrated to make a thermal model match a 100 W steady test. The same coefficient is then used to predict a 500 W transient.
1
The 100 W agreement is calibration evidence because h was chosen using that dataset.
2
It cannot independently validate the model or the 500 W transient prediction.
3
Collect separate tests spanning flow, temperature, and transient regimes with measured boundary conditions.
4
Assess whether a constant h model remains physically adequate or only compensates for omitted contact and radiation effects.
Result. Calibration can improve fit while hiding model discrepancy. Independent, relevant validation and uncertainty analysis are needed for the 500 W decision.
Independent check. Calibration and validation datasets, parameters, objectives, uncertainty, and applicability domains are explicitly separated.
Common misconceptions
Misconception
Correction
A calibrated model is validated
Calibration shows a fit under its data and assumptions; predictive validation requires relevant independent evidence.
A tool output closes the question
A result remains a candidate until its inputs, method, configuration, uncertainty, and relevance have been checked.
Diagnostic questions
Can aleatory and epistemic categories change?
Yes. The classification depends on the model, information, and decision context; what is unknown today may be measured tomorrow.
What would make this work reproducible?
Controlled inputs, method or code, versions, assumptions, outputs, and a stated interpretation tied to the decision.
Practice ladder
Basic
Classify four datasets as calibration, validation, or operational monitoring.
Intermediate
Design a validation matrix across heat load, flow, and ambient temperature.
Advanced
Explain parameter compensation and identifiability when contact resistance and convection are both tuned.
AI-assisted engineering task
Ask AI to identify whether each dataset influenced parameter fitting and to flag leakage between calibration and validation.
How to prove the AI output yourself
Inspect code and data splits.
Re-run calibration with held-out cases.
Check sensitivity and parameter identifiability.
Review experimental uncertainty and boundary measurements.
Retrieval and spaced review
Answer closed-notes today, then again after 1, 3, 7, and 30 days.
Define Calibration.
Inference of parameter values or distributions from observations and a model.
What role does Validation experiment play here?
An experiment designed to measure quantities and conditions relevant to assessing model adequacy.
What must a reviewer be able to reconstruct?
Calibration and validation datasets, parameters, objectives, uncertainty, and applicability domains are explicitly separated.
End-of-lesson summary
Calibration estimates model parameters using data under stated assumptions. Validation evaluates predictive adequacy using relevant, preferably independent data. Aleatory variability and epistemic uncertainty require different interpretations even when both are represented probabilistically.
Student notes
Label every dataset by role before fitting: calibration, validation, monitoring, or final decision evidence.
Use a visibly excellent calibrated fit and a poor held-out prediction. Students should distrust fit quality without data provenance.
7.4
Context of use, model risk, and credibility claims
Why this lesson matters
Credibility is not an intrinsic badge on a model. Required evidence depends on how the model influences a decision and the consequence of a wrong decision.
Learning objectives
Define and distinguish Context of use and Model influence.
Apply the lesson method to the worked context of use, model risk, and credibility claims case.
Evaluate evidence, uncertainty, and AI-assisted output before making a claim.
Readiness check
Before continuing, explain what decision this topic supports and name one upstream source that must be controlled.
Check your response
A sound answer names a specific engineering decision, its configuration, and a controlled requirement, model, dataset, interface, or standard that constrains the work.
Core idea
A context of use states the specific role and scope of a model in a decision. Risk-informed credibility scales verification, validation, and UQ rigor with model influence and decision consequence. The result is a bounded credibility claim with limitations, not 'the model is validated'.
Key concepts
Context of use
A precise statement of how and for what decision a model will be used.
Model influence
The degree to which the model determines or supports the decision.
Decision consequence
The severity of adverse outcome if the model contributes to a wrong decision.
Credibility claim
A bounded statement of confidence supported by specified evidence for a context of use.
Step-by-step explanation
State the question of interest and decision owner.
Write the model's context of use, including outputs, conditions, and role.
Assess model influence and consequence to determine model risk.
Set credibility goals and plan verification, validation, UQ, review, and reporting.
Compare achieved evidence with goals and issue a decision-specific claim and limitations.
Worked example
The beam model may be used in two ways: A, rank three concepts before prototype; B, certify that a flight-critical support meets its final deflection limit without physical testing.
1
Use A has moderate influence and limited consequence because later test can catch error; analytical checks and bounded comparisons may suffice.
2
Use B gives the model high influence and high consequence; substantially stronger configuration control, verification, validation, uncertainty, independent review, and justification are required.
3
Do not transfer evidence from A automatically to B because context, quantity, design maturity, and consequence differ.
4
Write separate credibility claims and unresolved limitations.
Result. The same code can be adequate for concept ranking and inadequate for certification. Credibility belongs to the model-use-evidence combination.
Independent check. The evidence plan is commensurate with influence and consequence, and the final claim never exceeds the validated conditions or quantities.
Common misconceptions
Misconception
Correction
A model is validated once
Validation evidence is conditional on quantity, domain, configuration, and context of use.
A tool output closes the question
A result remains a candidate until its inputs, method, configuration, uncertainty, and relevance have been checked.
Diagnostic questions
Can a low-fidelity model be credible?
Yes for a bounded low-risk use if evidence shows it is adequate for that decision.
What would make this work reproducible?
Controlled inputs, method or code, versions, assumptions, outputs, and a stated interpretation tied to the decision.
Practice ladder
Basic
Write a one-sentence context of use for concept ranking.
Intermediate
Compare credibility goals for ranking, sizing, and certification uses.
Advanced
Build a model-credibility assessment table with goals, achieved evidence, gaps, and residual decision risk.
AI-assisted engineering task
Ask AI to critique whether a credibility claim exceeds its context of use and cited evidence.
How to prove the AI output yourself
Read the full evidence, not the summary.
Check validated domain and configuration.
Compare planned and achieved credibility activities.
Require accountable decision-owner acceptance.
Retrieval and spaced review
Answer closed-notes today, then again after 1, 3, 7, and 30 days.
Define Context of use.
A precise statement of how and for what decision a model will be used.
What role does Model influence play here?
The degree to which the model determines or supports the decision.
What must a reviewer be able to reconstruct?
The evidence plan is commensurate with influence and consequence, and the final claim never exceeds the validated conditions or quantities.
End-of-lesson summary
A context of use states the specific role and scope of a model in a decision. Risk-informed credibility scales verification, validation, and UQ rigor with model influence and decision consequence. The result is a bounded credibility claim with limitations, not 'the model is validated'.
Student notes
Never write 'validated model' alone. Complete the sentence: validated for which quantity, conditions, configuration, and decision role?
Note that NASA-STD-7009B is the current edition and supersedes the A edition named in the original course brief. Use the public historical A text for comparison, but teach current status accurately.
LAB 7
Lab 7: Propagate uncertainty through a beam-deflection model
Lab objective
Use Monte Carlo sampling to propagate load, modulus, width, and height uncertainty and estimate the probability of exceeding a deflection limit.
Engineering context
The simply supported aluminum beam has a 2.50 mm deflection requirement. Height uncertainty is especially important because deflection scales with h^-3.
Input data
P = 1200 +/- 30 N
E = 69 +/- 1.5 GPa
b = 40.0 +/- 0.2 mm
h = 30.0 +/- 0.15 mm
L = 0.8 m
limit = 2.50 mm
Step-by-step task
Sample independent normal inputs as a teaching assumption
Compute I and deflection
Summarize mean and 95% interval
Estimate exceedance probability and rank correlations
Python code
import numpy as np
rng = np.random.default_rng(17)
n = 100_000
P = rng.normal(1200.0, 30.0, n)
E = rng.normal(69e9, 1.5e9, n)
b = rng.normal(0.0400, 0.0002, n)
h = rng.normal(0.0300, 0.00015, n)
L = 0.8
I = b * h**3 / 12.0
delta_mm = P * L**3 / (48.0 * E * I) * 1000.0
low, high = np.quantile(delta_mm, [0.025, 0.975])
print(f"mean={delta_mm.mean():.3f} mm")
print(f"95% interval=[{low:.3f}, {high:.3f}] mm")
print(f"P(delta > 2.50 mm)={(delta_mm > 2.50).mean():.5f}")
for name, values in {"P": P, "E": E, "b": b, "h": h}.items():
print(name, np.corrcoef(values, delta_mm)[0, 1])
Explanation of code
Step 1 sample independent normal inputs as a teaching assumption Step 2 compute I and deflection Step 3 summarize mean and 95% interval Step 4 estimate exceedance probability and rank correlations
Expected output
A mean near the nominal 2.06 mm, a finite 95% interval, and a small exceedance estimate that is reproducible with seed 17.
Interpretation
The calculation illustrates propagation, not a validated probability model. Distribution choice, correlation, tolerances, model form, and manufacturing data require justification.
Common errors
Treating +/- values as normal standard deviations without justification
Ignoring correlation
Reporting Monte Carlo precision as physical certainty
Extension tasks
Use bounded or tolerance-derived distributions
Add model discrepancy
Converge the exceedance estimate and compute sampling confidence
Reflection questions
Why does height dominate?
What uncertainty is missing?
Would a nominal pass be sufficient for a high-consequence decision?
WEEK 7
Weekly quiz and concept check
Closed notes. Answer each item, then use the key to correct in a different color.
What does code verification ask?
What does solution verification ask?
What does validation ask?
Why is calibration not validation?
What determines credibility rigor?
What is a defensible credibility claim?
Answer key
1. Whether software correctly implements the intended mathematical method.
2. How much numerical error remains in a particular solution.
3. Whether the model is adequately representative of reality for a defined use.
4. The same data used to tune parameters cannot independently test prediction.
5. Context of use, model influence, decision consequence, and residual uncertainty.
6. A bounded statement tied to quantities, conditions, configuration, evidence, use, and limitations.