Optimization · Module 2 of 10
Unconstrained Optimality Conditions
How do you know a point is a minimum? The gradient must vanish, and the Hessian must curve upward. These two conditions certify an answer and tell every descent method when to stop.
Readiness check
This module certifies a minimum. Tick only what you can do closed-notes.
- Take partial derivatives of a function of several variables.
- Recall that a maximum or minimum has zero slope.
- Evaluate a 2 by 2 determinant.
- Recall the second-derivative test in one variable.
- Solve two linear equations for two unknowns.
The core idea
At an unconstrained minimum the gradient is zero (first-order condition) and the Hessian is positive definite (second-order condition). The gradient locates candidate points; the Hessian classifies them as minima, maxima, or saddles.
first-order: ∇f(x*) = 0second-order: ∇2f(x*) positive definitestationary point ≠ minimum until the Hessian confirms itFor a smooth function with no constraints, calculus certifies an optimum. The gradient ∇f, the vector of partial derivatives, points in the direction of steepest increase; at any minimum or maximum it must vanish, so ∇f(x*) = 0 is the necessary first-order condition. A point where the gradient is zero is a stationary point, but it might be a minimum, a maximum, or a saddle. The Hessian ∇2f, the matrix of second partial derivatives, settles which: if it is positive definite at the stationary point, the function curves upward in every direction and the point is a strict local minimum; if negative definite, a maximum; if it has mixed signs, a saddle. For a 2 by 2 Hessian, a positive determinant with a positive leading entry means positive definite. When the function is convex, its Hessian is positive semidefinite everywhere, so any stationary point is a global minimum, the property that makes convex problems reliably solvable.
The skills, taught in order
Five skills build the gradient, the stationary point, the Hessian test, and convexity.
2.1 The gradient
The gradient ∇f collects the partial derivatives into a vector that points uphill, in the direction of steepest increase, with length equal to the steepest slope. Its negative points downhill, the direction every descent method follows. Where f is flat, the gradient is zero.
2.2 Stationary points and the first-order condition
The necessary first-order condition for an unconstrained optimum is ∇f(x*) = 0. Solving this system finds the stationary points, the only candidates for a minimum. It is necessary but not sufficient: a saddle also satisfies it.
2.3 The Hessian
The Hessian ∇2f is the symmetric matrix of second partial derivatives. It describes the local curvature: how the gradient changes as you move. Its definiteness at a stationary point classifies the point.
| Hessian at x* | Curvature | Point is |
|---|---|---|
| positive definite | up in all directions | local minimum |
| negative definite | down in all directions | local maximum |
| indefinite | mixed | saddle point |
| semidefinite | flat in some direction | inconclusive |
The second-order test. For a 2 by 2 Hessian, positive definite means a positive determinant and a positive top-left entry.
2.4 The second-order conditions
At a strict local minimum the gradient is zero and the Hessian is positive definite. A positive definite Hessian guarantees the function rises in every direction away from the point, which is exactly what a minimum requires. This pair of conditions certifies the answer.
2.5 Convexity
A function is convex if its Hessian is positive semidefinite everywhere. For a convex function any stationary point is a global minimum, so a zero gradient is enough. Convexity is the property that makes an optimization problem reliably and globally solvable.
Engineering connection: every descent method stops when the gradient is (nearly) zero, and the Hessian check is what distinguishes a genuine optimum from a saddle in a design space.
Worked example 1: minimum of a quadratic in two variables
Find the stationary point of f(x, y) = x2 + xy + y2 − 3x and classify it using the Hessian.
- ProblemFind and classify the stationary point of f in Figure 1.
- Given / findf(x, y) = x2 + xy + y2 − 3x. Find the stationary point and its type.
- AssumptionsSmooth unconstrained function; classify with the Hessian.
- ModelSolve ∇f = 0 for the candidate, then test the Hessian's definiteness.
- Equations∇f = (2x + y − 3, x + 2y)H = [[2, 1], [1, 2]]
- SolveFrom x + 2y = 0, x = −2y; substitute into 2x + y − 3 = 0: −4y + y − 3 = 0, y = −1, x = 2. So the stationary point is (2, −1). det H = 4 − 1 = 3 > 0 with top entry 2 > 0, so H is positive definite: a minimum. f(2, −1) = 4 − 2 + 1 − 6 = −3.
- CheckThe Hessian is constant (the function is quadratic), so the single stationary point is the global minimum. Both eigenvalues of H, 1 and 3, are positive, confirming positive definiteness.
- ConclusionThe point (2, −1) is the global minimum, f = −3, certified by a zero gradient and a positive definite Hessian.
Worked example 2: classifying stationary points in one variable
Find and classify the stationary points of f(x) = x3 − 3x using the first and second-order conditions.
- ProblemFind and classify the stationary points of f in Figure 2.
- Given / findf(x) = x3 − 3x. Find the stationary points and their types.
- AssumptionsSmooth one-variable function; use the second-derivative test.
- ModelSolve f′(x) = 0, then evaluate f″(x) at each root.
- Equationsf′(x) = 3x2 − 3f″(x) = 6x
- Solvef′ = 0 ⇒ x2 = 1 ⇒ x = ±1. At x = 1: f″ = 6 > 0, a minimum, f = 1 − 3 = −2. At x = −1: f″ = −6 < 0, a maximum, f = −1 + 3 = 2.
- CheckThe signs of f″ match the shape: the curve turns up at x = 1 and down at x = −1. Neither is global, since a cubic runs to ±∞, a reminder that these are local results.
- Conclusionx = 1 is a local minimum (f = −2) and x = −1 a local maximum (f = 2). The first-order condition alone could not tell them apart; the second-order test did.
Misconceptions and diagnostics
| Mistake | Symptom | Diagnostic question | Correction |
|---|---|---|---|
| Stationary means minimum | A saddle called an optimum | "Did I check the Hessian?" | A zero gradient is necessary, not sufficient. |
| Skipping the second-order test | Maximum mistaken for minimum | "Is the curvature up or down?" | Test the Hessian or f″ to classify. |
| Local taken as global | A local min reported as the answer | "Could another point be lower?" | Only convexity makes a local min global. |
| Sign error in the Hessian test | Definiteness judged wrong | "Is det > 0 and the top entry > 0?" | Both are needed for positive definite in 2 by 2. |
Practice ladder
Find the stationary point of f(x, y) = x2 + y2 − 2x − 6y + 14.
Show answer
∇f = (2x − 2, 2y − 6) = 0 ⇒ x = 1, y = 3. f(1, 3) = 1 + 9 − 2 − 18 + 14 = 4.
Classify that point using the Hessian.
Show answer
H = [[2, 0], [0, 2]], det = 4 > 0 with a positive top entry: positive definite, so a minimum.
Classify the stationary point of f(x, y) = x2 − y2.
Show answer
∇f = (2x, −2y) = 0 at (0, 0). H = [[2, 0], [0, −2]], det = −4 < 0: indefinite, so a saddle point.
A design objective has several stationary points found by a solver. Explain how you would tell a genuine minimum from a saddle without trusting the solver's label.
What good work looks like
Evaluate the Hessian (or its eigenvalues) at each stationary point: positive definite confirms a minimum, indefinite reveals a saddle. Certifying with the second-order condition is independent of what the solver reported.
Working with AI, and proving it yourself
Use AI as an examiner, not a solver
Portfolio task
Take a two-variable objective, find its stationary points, classify each with the Hessian, and state which is the design minimum and why.
Retrieval and spaced review
Closed notes. Answer out loud, then reveal.
1. State the first-order condition.
∇f(x*) = 0 at any unconstrained optimum.
2. What does the Hessian tell you?
The local curvature, which classifies a stationary point.
3. What certifies a strict minimum?
Zero gradient and a positive definite Hessian.
4. When is a 2 by 2 Hessian positive definite?
When its determinant and its top-left entry are both positive.
5. Why does convexity matter?
For a convex function any stationary point is the global minimum.
Textbook mapping
This module follows Nocedal and Wright, Numerical Optimization, 2nd edition. Use these references to read further.
| Topic in this module | Where to read more |
|---|---|
| Gradient and first-order conditions | Nocedal & Wright, Chapter 2 |
| Hessian and second-order conditions | Nocedal & Wright, Chapter 2 |
| Convexity | Nocedal & Wright, Chapter 2 |
Chapter numbers refer to the 2nd edition. The optimality conditions are standard, so any recent edition will align closely.