Math for ME · Chapter 19 of 19 · Advanced
Engineering Optimization
Design is rarely about finding an answer that works; it is about finding the best one that works. Optimization is the mathematics of "best", whether that means the lightest part, the cheapest tank, or the fastest settling controller.
The thread: You can now model a system and solve it. The last step of design is choosing the best solution the constraints allow, and that is what optimisation finds.
Readiness check
This chapter draws on Multivariable Calculus and Numerical Methods. Tick only what you can do closed-notes.
- Find a maximum or minimum by setting a derivative to zero (Derivatives).
- Take partial derivatives and write a gradient (Multivariable Calculus).
- Use a Lagrange multiplier idea: ∇f parallel to ∇g (Multivariable Calculus).
- March a quantity forward in small steps (Numerical Methods).
- Substitute a constraint to remove a variable.
The core idea
Write down what you want to make best, the knobs you can turn, and the limits you must respect. Then move toward the best the knobs allow.
minimise f(x) subject to g(x) = 0Every optimisation has three parts: an objective f (cost, mass, error), the design variables x (the knobs), and the constraints g (volume, strength, budget). If there are no constraints, the best sits where the gradient is zero. If constraints bind, the best sits on the boundary, where the objective can fall no further without breaking a limit. The rest of the chapter is three ways to find that point.
The skills, taught in order
19.1 Naming the problem
Half of optimisation is stating it cleanly: the objective to minimise or maximise, the design variables you control, and the constraints that fence the design in. Get these three wrong and no method helps. A weight problem with strength left out simply returns "use no material".
19.2 Unconstrained: set the gradient to zero
With no binding constraints, an interior optimum sits where the objective stops sloping in every direction, that is where the gradient vanishes:
∇f = 0Solve for the candidate points, then use the second derivative to tell a minimum from a maximum or a saddle, exactly the test from Multivariable Calculus. Stationary does not mean optimal until that check is done.
19.3 Constrained: Lagrange multipliers
When a constraint binds, the optimum lies where the objective's contours just touch the constraint, so their gradients are parallel:
∇f = λ∇gThe multiplier λ is not just bookkeeping; it is a sensitivity, the rate at which the best objective would improve if the constraint were relaxed by one unit. Engineers read it as a shadow price: how much a little more material, budget, or volume would buy you.
19.4 Numerical: gradient descent
When the objective has no clean formula, march downhill. Gradient descent steps against the gradient, the direction of steepest decrease, by a chosen step size α:
xnext = x − α ∇f(x)Too small a step crawls; too large a step overshoots and can diverge, the same stability limit seen in Numerical Methods. This single rule, scaled up, trains machine-learning models and drives most design-optimisation software.
19.5 Linear programming
When the objective and all constraints are linear, the feasible region is a polygon and the optimum always sits at one of its corners. Linear programming exploits that to allocate limited resources, material, machine time, power, among competing demands. It is the backbone of production planning and logistics.
Engineering connection: design optimisation, structural and FEA shape optimisation, control tuning, resource allocation, and the mathematics behind machine learning.
Worked example: the cheapest tank when the ends cost more
A closed cylindrical tank must hold V = 1.0 m³. The flat end caps cost twice as much per square metre as the rolled wall. Find the radius and height that minimise total cost, and compare the shape with the equal-cost case.
- ProblemMinimise cost for the tank in Figure 1 and read off the best shape.
- Given / findClosed cylinder, volume πr²h = 1.0 m³; end caps cost 2 per m², wall costs 1 per m². Find r and h.
- Objectivecost C = (caps) + (wall) = 2(2πr²) + 1(2πrh) = 4πr² + 2πrh.
- Use the constraint to remove a variableh = V/(πr²), so C(r) = 4πr² + 2V/r.
- Set the derivative to zerodC/dr = 8πr − 2V/r² = 0, giving r³ = V/(4π).
- Solver = (1.0/4π)1/3 = 0.430 m, and h = V/(πr²) = 1.72 m, so h = 4r.
- Confirm a minimumd²C/dr² = 8π + 4V/r³ > 0, so the cost is genuinely minimised.
- Check against equal costwhen caps and wall cost the same, the optimum is h = 2r. Doubling the cap cost pushed the tank to h = 4r, taller and narrower to shrink the expensive end area. The maths matched the intuition.
- ConclusionObjective, constraint, derivative, second-derivative check: the same routine that sized parts in calculus now trades real costs against geometry. The optimal shape moves with the price ratio, which is the whole point of optimisation.
Worked example 2: marching downhill by gradient descent
A cost surface is f(x, y) = (x − 4)² + (y − 3)², whose minimum is plainly at (4, 3). Pretend you cannot see that and instead run gradient descent from (0, 0) with step size α = 0.2, to show how the method finds it.
- Given / findf(x, y) = (x − 4)² + (y − 3)², start (0, 0), α = 0.2. Take two steps and show the trend.
- Gradient∇f = (2(x − 4), 2(y − 3)): it points uphill, so we step the opposite way.
- Step rule(x, y)next = (x, y) − α ∇f
- Step 1at (0, 0), ∇f = (−8, −6). New point: (0, 0) − 0.2(−8, −6) = (1.6, 1.2). Cost drops from 25 to 9.
- Step 2at (1.6, 1.2), ∇f = (−4.8, −3.6). New point: (1.6, 1.2) − 0.2(−4.8, −3.6) = (2.56, 1.92). Cost falls to 3.24.
- Read the patterneach coordinate closes 40% of its remaining gap to the target every step (the rule reduces to x ← 0.6x + 1.6), so the path converges steadily to (4, 3).
- Check the step sizewith α this small the march is stable. Far too large a step would overshoot the valley and the cost would grow, the same stability limit as marching an ODE in Numerical Methods.
- ConclusionFollow the negative gradient, in small steps, and a complicated cost surface is searched without ever solving ∇f = 0 by hand. This is the engine inside structural-shape optimisation and machine learning alike.
Misconceptions and diagnostics
| Mistake | Symptom | Diagnostic question | Correction |
|---|---|---|---|
| Stationary mistaken for optimal | A saddle or maximum reported as the best design | "Did the second-derivative test confirm a minimum?" | ∇f = 0 only finds candidates. Confirm with the second derivative before trusting it. |
| Constraints left out | The optimiser returns zero material or infinite size | "What stops this design from shrinking forever?" | Every real problem has binding limits. State the strength, volume, or budget constraint explicitly. |
| Gradient-descent step too large | The cost rises instead of falling | "Is my step size small enough for this surface?" | Reduce α. An overshooting step diverges, just like an oversized step in a numerical solver. |
| Local optimum taken as global | A different start gives a better answer | "Did I try more than one starting point?" | On a bumpy landscape, restart from several places and keep the best; a single descent only finds a local dip. |
Practice ladder
Minimise f(x) = x² − 6x + 11 and confirm it is a minimum.
Show answer
f′ = 2x − 6 = 0 gives x = 3; f″ = 2 > 0, a minimum. The least value is f(3) = 2.
Then take one gradient-descent step on f(x) = (x − 5)² from x = 0 with α = 0.3.
Show answer
f′(0) = 2(0 − 5) = −10. xnext = 0 − 0.3(−10) = 3. One step moved 60% of the way to the minimum at 5.
A rectangular pen of fixed perimeter 40 m is built against a wall (only three sides fenced). Maximise the area.
Show answer
With sides x (two of them) and y, fence 2x + y = 40, area A = xy = x(40 − 2x). A′ = 40 − 4x = 0 gives x = 10, y = 20, A = 200 m². The classic result: the wall side is twice the others.
For the tank example, what would the Lagrange multiplier λ tell a designer?
Show answer
It is the sensitivity of the minimum cost to the volume requirement: roughly how much extra cost each additional cubic metre of capacity would add. A shadow price for volume.
Minimise the surface area of an open-top box (no lid) of square base, side a and height h, holding 4 litres. Set it up and solve.
Show answer
Volume a²h = 4000 cm³, area A = a² + 4ah = a² + 16000/a. A′ = 2a − 16000/a² = 0 gives a³ = 8000, a = 20 cm, h = 10 cm, A = 1200 cm². Height is half the base side once the lid is removed.
Take a real design with a clear trade-off (a bracket's weight versus stiffness, a fin's heat loss versus material, a gear ratio's speed versus torque). Write the objective, the design variable, and the binding constraint, then find or estimate the optimum and state what the answer reveals about the trade-off.
What good work looks like
A named objective and constraint, the optimum found by calculus or a short gradient-descent run, and a sentence such as "below this thickness stiffness falls too fast, above it weight grows with no real gain."
Working with AI, and proving it yourself
Use AI as an examiner, not a solver
Portfolio task
Take one real component and run a small optimisation end to end: state the objective and constraint, solve it by calculus, then reach the same answer by a few gradient-descent steps in a spreadsheet, and report the optimal design with the trade-off it reveals.
Retrieval and spaced review
Closed notes. Answer out loud, then reveal.
1. What three parts define any optimisation problem?
The objective to make best, the design variables you control, and the constraints that limit the design.
2. Where is an unconstrained optimum, and how do you confirm it?
Where the gradient is zero, confirmed as a minimum by the second-derivative test.
3. State the Lagrange condition and what the multiplier means.
∇f = λ∇g at a constrained optimum; λ is the sensitivity of the best objective to relaxing the constraint, a shadow price.
4. Write the gradient-descent step and its risk.
xnext = x − α∇f. Too large an α overshoots and diverges; too small crawls.
5. Where does a linear program's optimum sit?
At a corner (vertex) of the feasible polygon, because both objective and constraints are linear.
Textbook mapping
| Item | Mapping |
|---|---|
| Main source | Kreyszig, Advanced Engineering Mathematics, Ch 22 (unconstrained optimisation, linear programming). Design companion: any engineering optimisation or design text |
| Core topics | 19.1 Naming the problem · 19.2 Unconstrained optima · 19.3 Lagrange multipliers · 19.4 Gradient descent · 19.5 Linear programming |
| Engineering connection | Design optimisation, structural and FEA shape optimisation, control tuning, resource allocation, and the optimisation core of machine learning. |
| Skip on first pass | Karush-Kuhn-Tucker conditions in full, the simplex algorithm's internals, convexity theory, and stochastic optimisation. |
| Read next | The math course is complete. Carry these tools into Statics and the engineering courses ahead. |