Digital Engineering Foundations · Module 5 of 8
Simulation, Verification and Evidence
A simulation result is not automatically evidence. Learn how analysis and test become verification evidence, why validation and applicability limit what a result can claim, and how credibility, uncertainty awareness and model-result traceability keep the claim honest.
Readiness check
Building on Modules 1–4. Tick only what you can do closed-notes.
- Explain the difference between a calculation and a physical test.
- Identify a requirement that inspection can verify.
- State the assumptions and configuration behind a model result (from Module 4).
- Describe the evidence needed for one design claim.
- Explain why a result may not apply after a geometry change.
The core idea
Verification asks whether specified requirements have been met; validation asks whether the system satisfies stakeholder use in context. A result becomes evidence only when its method, assumptions, applicability and credibility are recorded.
Evidence can come from inspection, analysis, demonstration or test, chosen to fit the requirement and its consequence. A simulation can support verification or prediction, but it does not validate a design by itself, validation needs the real use context. This module keeps the distinctions clear and practical; the detailed methods of code and solution verification, validation metrics and uncertainty quantification belong to the dedicated VVUQ course, and measurement uncertainty to Measurements.
The skills, taught in order
Four steps: turn simulation into evidence, choose verification methods, respect validation and applicability, and keep credibility and traceability with the result.
5.1 Simulation and analysis as evidence
A simulation result is a model output; it becomes evidence when it is tied to the requirement it addresses, the model purpose and configuration from Module 4, and an acceptance criterion set in advance. Maximum stress can verify a strength requirement by a checked analysis only if model scope, assumptions and configuration are recorded. The result without that context is a number, not an inspectable claim.
5.2 Verification methods and their evidence
Verification asks whether specified requirements have been met, using the method that fits the requirement and risk: inspection (hole spacing against a drawing), analysis (a checked stress calculation), demonstration (the mechanism actuates), or test (a load to failure). Choose a method for each requirement, state acceptance criteria before collecting evidence, and link each evidence item to its requirement and configuration. Not every requirement needs a physical test; choose methods that fit the requirement and consequence.
5.3 Validation, applicability and uncertainty awareness
Validation asks whether the system satisfies its intended use in context, a different question from verification. Every result has an applicability domain: the conditions, geometry, loads and assumptions under which it is meaningful. Reusing a result outside that domain without review is the most common quiet error. Treat every number as having uncertainty from inputs, model form and measurement; you do not need full uncertainty quantification here, but you must state the main sources and avoid overclaiming. The formal methods live in the VVUQ course; measurement uncertainty in Measurements.
5.4 Credibility, evidence records and model-result traceability
Credibility is the recorded basis for trusting a result: what was verified, against what data, with which assumptions and known limits. An evidence record ties the result to its model configuration, inputs, method, acceptance criterion and reviewer. Model-result traceability means a decision can be walked back from the claim, through the evidence, to the model and its inputs, the Module 2 provenance discipline applied to results. This is what lets a reviewer accept a simulation as verification rather than taking it on faith.
Course scope. This module is deliberately foundational. For detailed code/solution verification, validation metrics, sensitivity and uncertainty quantification, continue to VVUQ; for calibration and measurement uncertainty, Measurements.
Worked example 1: choosing verification methods
Assign a verification method and acceptance criterion to each bracket requirement, and flag any that need physical test evidence.
- ProblemMatch four requirements to inspection, analysis, demonstration or test, with acceptance criteria.
- Given / findRequirements: mass ≤ 300 g; static stress ≤ 0.6·yield at 600 N; fatigue survives 10,000 cycles at 600 N; fastener pattern two M6 at 50 mm pitch. Find methods + criteria.
- ModelChoose the lowest-cost method that gives sufficient evidence for the requirement's consequence; state the pass criterion before testing.
- SolveMass → inspection (weigh; pass if ≤ 300 g). Static stress → analysis (checked FEA on the released configuration; pass if peak ≤ 0.6·yield, with assumptions recorded). Fatigue → test (cyclic load rig; pass if no crack at 10,000 cycles), physical evidence is needed because fatigue depends on surface, residual stress and real geometry the model idealizes. Fastener pattern → inspection against the drawing.
- Check"Every requirement needs a test" is false, inspection and analysis suffice for three of four. "Simulation validates the design" is avoided: the analysis verifies the static requirement, it does not validate field use.
- ConclusionMethod choice is an engineering judgment about evidence sufficiency and consequence, recorded before evidence is collected.
Worked example 2: is this simulation admissible evidence?
An FEA reports peak stress 0.55·yield and the team wants to close the static requirement. Decide whether it is admissible verification evidence.
- ProblemJudge admissibility on applicability and credibility, not on the headline number.
- Given / findFEA peak stress = 0.55·yield (criterion ≤ 0.60). Recorded: mesh unspecified, load = 600 N, geometry = an earlier CAD revision, material default library value, no hand check. Find whether it verifies REQ-LOAD on baseline B1.
- ModelAdmissible = on the current configuration, within the model's applicability, with recorded credibility (mesh/assumptions/data) and an independent sanity check.
- SolveApplicability fails on configuration: geometry is an earlier revision than B1 → the result does not apply to the released design. Credibility is thin: mesh convergence unrecorded, material value unverified, no hand check. Even though 0.55 < 0.60, the evidence cannot close the requirement as-is.
- CheckThe margin (0.55 vs 0.60) is small enough that mesh and material uncertainty could cross it, exactly why credibility must be recorded rather than assumed.
- ConclusionRe-run on the B1 geometry with a convergence check, a verified material property and an order-of-magnitude hand check, then record the evidence with its configuration. A number that passes is not yet evidence that verifies.
Misconceptions and diagnostics
| Mistake | Diagnostic question | Correction |
|---|---|---|
| Simulation validates the design | "Was physical use in context tested?" | Simulation supports verification or prediction; validation needs the real context. |
| Every requirement needs a test | "Can inspection or analysis provide enough evidence?" | Choose the method that fits the requirement and its consequence. |
| A passing number is evidence | "On which configuration, with what credibility?" | A result becomes evidence only with applicability and a credibility record. |
| Results are context-free | "For which question and configuration was it produced?" | Outputs stay tied to purpose, applicability and configuration. |
| Uncertainty can be ignored if the margin looks fine | "Could input, model or mesh uncertainty cross the criterion?" | State the main uncertainty sources; a small margin needs recorded credibility. |
Practice ladder
Assign a verification method to each final-project requirement, write acceptance criteria for three of them, and identify one requirement needing physical test evidence.
Show answer
Methods fit the requirement (inspection/analysis/demonstration/test); criteria are set before evidence; the test-only requirement is typically fatigue, wear, or a failure mode the model idealizes.
Take one analysis result and write its evidence record: model configuration, inputs, assumptions, method, acceptance criterion, and the main uncertainty sources.
Show answer
The record ties the result to a configuration and criterion, names the assumptions that control it, and lists uncertainty sources (inputs, model form, mesh, material data) without pretending to full quantification.
Given a simulation result on an earlier revision, decide whether it is admissible for the current baseline and state exactly what must change for it to count.
Show answer
Good work rejects it on applicability (configuration) and/or credibility (mesh, material, no independent check), and lists the specific fixes: current geometry, convergence check, verified data, and a hand check.
For one real requirement, produce a complete evidence record and a two-line credibility statement, linking the result back to its model and inputs.
What good work looks like
An evidence record with method, criterion, configuration and uncertainty sources, and full model-result traceability, the verification layer of the Module 8 capstone. Deeper VVUQ methods are the next course.
Working with AI, and proving it yourself
Use AI as an examiner, not a solver
Portfolio task
Draft an evidence record with AI, then verify every field, method, criterion, configuration, uncertainty sources, against the actual analysis or test before it counts as verification.
Retrieval and spaced review
Closed notes. Answer out loud, then reveal.
1. What is verification?
Checking whether specified requirements have been met.
2. What is validation?
Checking adequacy for the intended use or stakeholder need in context.
3. What is an applicability domain?
The conditions, geometry, loads and assumptions under which a result is meaningful.
4. When does a result become evidence?
When it is tied to a requirement, on the right configuration, with a method, acceptance criterion and recorded credibility.
5. Where do the detailed VVUQ and uncertainty methods live?
In the dedicated VVUQ and Measurements courses; this module stays foundational.