Digital Engineering Foundations · Module 7 of 8
AI-Assisted Digital Engineering
Use AI for bounded extraction, review and candidate links while accountable engineers keep authority. Ground every suggestion in sources, measure it with precision and recall, and keep an audit trail so AI accelerates work without replacing judgment.
Readiness check
Building on Modules 1–6. Tick only what you can do closed-notes.
- Explain a human review step and distinguish suggestion from approval.
- Identify a source document to ground an AI task.
- Compare a generated output with a source and record a reviewer decision.
- Compare two sets of links and count matches and misses.
- Explain why both wrong links and missing links matter.
The core idea
AI may extract, classify, generate or suggest; accountable engineers retain authority for acceptance and approval. Useful AI tasks in digital engineering are bounded, source-grounded, and measured.
Good tasks include requirements-quality review, metadata drafting, document classification, candidate trace links, log summarisation and comparison of structured records. Every output is a review aid: false positives waste time, false negatives leave defects hidden, so human review is part of the method, not an optional courtesy. Precision and recall describe how good the assistance is, and an audit record, prompt, sources, output, edits, reviewer, disposition, makes the whole workflow inspectable.
The skills, taught in order
Four steps: pick bounded tasks, use AI to review requirements and information, evaluate AI traceability with precision and recall, and govern the workflow with audit trails and human authority.
7.1 Appropriate AI tasks in digital engineering
Define the task boundary, provide sources and a required output schema, require abstention when evidence is missing, and assign human review and approval. Bounded, useful tasks include requirements-quality review, metadata drafting, document classification, candidate trace links, simulation-log summarisation and comparison of structured records. AI output is assistance, not engineering authority, if a relationship is wrong, an accountable person, not the model, is answerable.
7.2 AI-assisted requirements and information review
AI can inspect requirements for missing units, vague verbs, duplicated statements, inconsistent identifiers and metadata gaps. Provide the source text and quality criteria, ask for structured findings with evidence references, check each finding against the source, and record acceptance, rejection or edit. Absence of flags is not proof of quality (false negatives), and a flag is not automatically correct (false positives), keep before-and-after text so another engineer can inspect the human change.
7.3 AI-assisted traceability and change analysis
AI-generated trace links can accelerate review but must be evaluated against a verified reference or human review. Represent links as source-target-type, compare candidates with the reference, and compute true positives, false positives and false negatives. Precision = TP/(TP+FP) measures how many proposed links were correct; recall = TP/(TP+FN) measures how many real links were found. High precision with low recall misses evidence; high recall with low precision creates review noise. A plausible link is not a verified one.
7.4 Verification, governance and human authority
Practical AI governance reaches the working-artifact level: define the task, control inputs, require structured output and source links, test the output, record uncertainty, preserve the audit trail, and make human authority explicit. An audit record captures prompt, source files, tool/model, output, reviewer, accepted and rejected items, and downstream use. A confidence score is evidence only if it was defined and calibrated for the task. The aim is neither to fear nor to worship AI, but to use it without losing engineering accountability.
Lab & references. Lab 6 evaluates AI-generated trace-link candidates using stored outputs (no external API key needed). For governance, see the NIST AI Risk Management Framework.
Worked example 1: scoring AI trace links with precision and recall
AI proposes trace links; a verified reference set exists. Score the assistance and decide how to improve it.
- ProblemJudge AI link quality with precision and recall, not by "it looks plausible."
- Given / findReference (true) links: 8. AI proposed 9 links; of these, 6 match the reference (TP), 3 do not (FP); 2 reference links were missed (FN). Find precision, recall, and the improvement.
- ModelPrecision = TP/(TP+FP); recall = TP/(TP+FN). Interpret which error type dominates.
- SolvePrecision = 6/(6+3) = 6/9 = 0.67. Recall = 6/(6+2) = 6/8 = 0.75. The 3 false positives are review noise (e.g. a mass requirement linked to a stress result because both mention "bracket"); the 2 false negatives are missed evidence (e.g. a test procedure that verifies the load requirement in different wording).
- CheckAccuracy alone would hide this; precision and recall separate noise from missed evidence. The link type must match the evidence, plausibility is not verification.
- ConclusionTo lift precision, tighten the prompt/schema so links require a matching type; to lift recall, add synonym handling so differently-worded verification links are found. Then re-score.
Worked example 2: an AI requirements review, dispositioned
AI flags three requirement issues. Disposition each against the source and record the audit trail.
- ProblemTurn AI findings into reviewed, recorded engineering decisions.
- Given / findFindings: (1) "durable" is vague; (2) "mass ≤ 300 g" duplicates another requirement; (3) REQ-LOAD lacks a verification method. Source: the requirements register. Find each disposition.
- ModelEach finding is checked against the source and marked accepted, edited or rejected, with a reason and before/after text.
- Solve(1) Accepted + edited: replace "durable" with a fatigue criterion (10,000 cycles at 600 N). (2) Rejected: the source shows the "duplicate" is actually a different requirement (mass of a sub-bracket), a false positive. (3) Accepted: add "verify by analysis" to REQ-LOAD. Record prompt, sources, findings, dispositions and reviewer.
- Check"AI found it, so it is wrong" is avoided (finding 2 was a false positive); "AI missed nothing, so the file is clean" is avoided by keeping the review a human step regardless of flag count.
- ConclusionThe value is not the raw AI output but the reviewed, recorded dispositions that another engineer can inspect.
Misconceptions and diagnostics
| Mistake | Diagnostic question | Correction |
|---|---|---|
| AI approval is enough | "Who is accountable if the relationship is wrong?" | AI output is assistance, not engineering authority. |
| More context always fixes hallucination | "Can the model still invent unsupported links?" | Controls, source grounding and review are still required. |
| AI found it, so it is wrong / AI missed nothing, so it is clean | "Does the source support the finding? What false negatives were tested?" | Every finding needs review; absence of flags is not proof. |
| A plausible link is correct | "Does the link type match the evidence?" | Plausibility is not verification. |
| Confidence score means correctness | "Was it calibrated for this task?" | Confidence is evidence only if defined and tested. |
Practice ladder
Choose one bounded AI task for the final project, write its source inputs and output schema, write an abstention rule, and define the human approval step.
Show answer
A bounded task (e.g. requirements-quality review), named sources, a structured output schema, a rule to abstain when evidence is missing, and a named approver.
Run a requirement-quality review on three statements, accept one AI-style finding and reject one, edit one requirement manually, and record the disposition.
Show answer
Findings checked against the source, one accepted, one rejected (false positive), one edited, all with before/after text and a recorded reviewer.
Run the AI trace-evaluation lab, explain each false positive and false negative, compute precision and recall, and write one rule that would improve candidate links.
Show answer
Correct TP/FP/FN counts, precision and recall, an explanation of each error, and a schema/prompt rule targeting the dominant error type.
For one real AI-assisted task, produce a complete audit record and a two-line statement of what a human verified before use.
What good work looks like
An audit record (prompt, sources, output, dispositions, reviewer) plus a human-verification statement, the AI-governance layer of the Module 8 capstone.
Working with AI, and proving it yourself
Use AI as an examiner, not a solver
Portfolio task
Run one bounded AI task end to end, sources in, structured candidates out, human review, audit record, and report its precision and recall against a reference.
Retrieval and spaced review
Closed notes. Answer out loud, then reveal.
1. What may AI do, and who retains authority?
Extract, classify, generate or suggest; accountable engineers retain authority for acceptance and approval.
2. What are false positives and false negatives?
Suggested issues that are not real; and real issues the AI did not flag.
3. Define precision and recall.
Precision = TP/(TP+FP); recall = TP/(TP+FN).
4. What is source grounding?
Tying claims or suggestions to inspectable source artifacts.
5. What belongs in an AI audit record?
Prompt, source files, tool/model, output, reviewer, accepted and rejected items, and downstream use.