Home / Courses / Optimization / KKT Conditions

Optimization · Module 6 of 10

Constrained Optimization and the KKT Conditions

Real designs live on their limits. At a constrained optimum the gradient of the objective is balanced by the constraint gradients, weighted by Lagrange multipliers, and the KKT conditions make that balance precise.

Readiness check

This module adds constraints. Tick only what you can do closed-notes.

Compute a gradient of a function.
Recall that a constraint gradient is normal to its curve.
Solve a small system of equations.
Recall an active versus inactive constraint.
Recognise when an unconstrained minimum violates a limit.

0 or 1 weak itemsContinue with this module.

2 weak itemsRevisit optimality conditions in Module 2.

3 or more weak itemsRevisit formulation and active constraints in Module 1.

The core idea

At a constrained optimum the objective gradient is a combination of the active constraint gradients: ∇f = Σ λ_i ∇c_i. The KKT conditions collect this balance with feasibility, multiplier signs, and complementary slackness.

equality: ∇f = λ ∇hinequality: ∇f + μ ∇g = 0, μ ≥ 0complementary slackness: μ g = 0

Most engineering optima sit on a constraint boundary, not at a free minimum. For an equality constraint h(x) = 0, the method of Lagrange multipliers says that at the optimum the objective gradient is parallel to the constraint gradient: ∇f = λ∇h, because any feasible move along the constraint can no longer lower f. The scalar λ is the Lagrange multiplier, and it equals the rate at which the optimal objective would change if the constraint were relaxed, its shadow price. Inequality constraints g(x) ≤ 0 add more logic, captured by the Karush-Kuhn-Tucker (KKT) conditions: the point is feasible, the Lagrangian gradient vanishes, the inequality multipliers are non-negative, and complementary slackness μg = 0 holds, meaning a constraint is either active (g = 0 with μ ≥ 0) or inactive (μ = 0). These conditions are the constrained generalisation of a zero gradient and are what certifies a constrained design optimum.

The skill works when: you balance the gradients with multipliers and check every KKT condition.

The skill breaks down when: an inequality multiplier is allowed negative, or complementary slackness is ignored.

The concept. The optimum lies where an objective contour is tangent to the constraint. There the two gradients are parallel, and the multiplier is their ratio.

The skills, taught in order

Five skills build from equality constraints to the full KKT conditions.

6.1 Equality constraints and Lagrange multipliers

To minimise f subject to h(x) = 0, form the Lagrangian L = f − λh and set its gradient to zero. This yields ∇f = λ∇h together with the constraint, a solvable system whose multiplier λ measures the constraint's sensitivity.

6.2 The meaning of the multiplier

The Lagrange multiplier is the shadow price of the constraint: it equals the change in the optimal objective per unit relaxation of the constraint. A large multiplier flags a binding, costly constraint; a zero one flags a constraint that does not matter.

6.3 Inequality constraints

An inequality g(x) ≤ 0 is either active (g = 0, holding the solution) or inactive (g < 0, slack). At the optimum only the active constraints enter the gradient balance, each with a non-negative multiplier that pushes the solution inward.

KKT condition	Statement	Meaning
Stationarity	∇f + Σμ∇g + Σλ∇h = 0	gradients balance
Primal feasibility	g ≤ 0, h = 0	constraints hold
Dual feasibility	μ ≥ 0	inequality multipliers non-negative
Complementary slackness	μ g = 0	active or inactive, not both

The four KKT conditions. Together they are the necessary conditions for a constrained optimum.

6.4 The KKT conditions

The KKT conditions collect stationarity of the Lagrangian, feasibility, non-negative inequality multipliers, and complementary slackness. A point satisfying them is a candidate constrained optimum, the direct generalisation of ∇f = 0.

6.5 Complementary slackness

Complementary slackness μg = 0 says each inequality is either active with a positive multiplier or inactive with a zero multiplier. It is the logical switch that identifies which constraints bind, the heart of solving a constrained problem by cases.

Engineering connection: the multiplier on a stress or weight constraint is its shadow price, telling you exactly how much the design would improve if the limit were loosened.

Worked example 1: an equality-constrained minimum

Minimise f(x, y) = x² + y² subject to x + y = 1 using Lagrange multipliers. Find the optimum and the multiplier.

Figure 1. Minimising the squared distance to the origin subject to a line constraint gives the foot of the perpendicular. There the objective gradient aligns with the constraint normal.

ProblemMinimise f subject to the line constraint in Figure 1.
Given / findf(x, y) = x² + y², h(x, y) = x + y − 1 = 0. Find the optimum and λ.
AssumptionsThe equality holds; the Lagrange stationarity condition applies.
ModelSet ∇f = λ∇h with the constraint: 2x = λ, 2y = λ, x + y = 1.
Equations2x = λ, 2y = λx + y = 1
SolveFrom the first two, x = y = λ/2. The constraint gives x + y = λ = 1, so λ = 1 and x = y = 0.5. f = 0.25 + 0.25 = 0.5.
CheckThe point (0.5, 0.5) is the foot of the perpendicular from the origin to the line, the geometric minimum. The multiplier λ = 1 is the rate f would change if the line were shifted.
ConclusionThe constrained minimum is (0.5, 0.5), f = 0.5, with multiplier 1. Lagrange multipliers turn a constrained problem into a solvable system.

Result. Optimum (0.5, 0.5), f = 0.5, multiplier λ = 1.

Worked example 2: an active inequality and the KKT conditions

Minimise f(x, y) = (x − 2)² + (y − 2)² subject to x + y ≤ 2 (with x, y ≥ 0). Find the optimum and verify the KKT conditions.

Figure 2. The free minimum (2, 2) is infeasible, so the constraint is active. The optimum sits on the line x + y = 2, at the closest feasible point (1, 1).

ProblemFind the constrained minimum in Figure 2 and check the KKT conditions.
Given / findf = (x − 2)² + (y − 2)², g = x + y − 2 ≤ 0. Find the optimum and the multiplier μ.
AssumptionsThe free minimum (2, 2) violates the constraint, so it is active: g = 0.
ModelOn x + y = 2 minimise f; check ∇f + μ∇g = 0 with μ ≥ 0 and μg = 0.
Equations∇f = (2(x − 2), 2(y − 2)), ∇g = (1, 1)x + y = 2
SolveBy symmetry on the line, x = y = 1, f = 1 + 1 = 2. Then ∇f = (−2, −2), and ∇f + μ(1, 1) = 0 gives μ = 2 ≥ 0. Complementary slackness holds since g = 0.
CheckAll KKT conditions hold: feasibility (x + y = 2), stationarity (μ = 2 balances the gradients), dual feasibility (μ > 0), and complementary slackness (μg = 0). The constraint is genuinely active.
ConclusionThe optimum is (1, 1), f = 2, held on the boundary by an active constraint with multiplier 2. The KKT conditions certify it.

Result. Optimum (1, 1), f = 2, active constraint with μ = 2.

Misconceptions and diagnostics

Mistake	Symptom	Diagnostic question	Correction
Ignoring an active constraint	Free minimum reported as the answer	"Is the free optimum feasible?"	If it violates a constraint, that constraint is active.
Negative inequality multiplier	μ < 0 at the optimum	"Is μ ≥ 0?"	Inequality multipliers must be non-negative.
Skipping complementary slackness	Constraint both active and slack	"Is μg = 0?"	Each inequality is active or inactive, not both.
Sign of the multiplier confused	Wrong shadow-price interpretation	"Which way does relaxing help?"	The multiplier is the sensitivity of the optimum to the limit.

Practice ladder

Level 1 · Direct skill

Minimise x² + y² subject to x + y = 4. Find the optimum.

Show answer

By symmetry x = y = 2, with λ = 4 (from 2x = λ, x + y = 4). f = 8.

Level 2 · Mixed concept

For that problem, what is the multiplier's meaning?

Show answer

λ = 4 is the shadow price: raising the right-hand side from 4 to 5 would change the optimal f at the rate of about 4 per unit.

Level 3 · Independent problem

Minimise (x − 3)² + y² subject to x ≤ 1. Is the constraint active, and what is the optimum?

Show answer

The free min is (3, 0), which violates x ≤ 1, so the constraint is active: x = 1, y = 0, f = 4. The multiplier μ satisfies 2(1 − 3) + μ = 0, μ = 4 ≥ 0.

Transfer task | Real engineering

A weight-minimisation gives a large multiplier on the stress constraint. Explain what that tells the designer.

What good work looks like

The stress constraint is strongly binding: relaxing the allowable stress (say with a stronger material) would reduce weight at a high rate given by the multiplier, so it is the constraint most worth revisiting.

Working with AI, and proving it yourself

Use AI as an examiner, not a solver

"Check that all four KKT conditions hold at my candidate point."

"Give me three constrained problems; I will identify the active constraints."

"Solve the constrained problem for me." Setting up the Lagrangian is the skill.

"Is this the optimum?" Verifying the KKT conditions yourself is the point.

Portfolio task

Solve a real constrained design by Lagrange multipliers or KKT, identify which constraints are active, and interpret the multipliers as shadow prices.

Must include: the Lagrangian, the active set, the multipliers, and a shadow-price reading.

Retrieval and spaced review

Closed notes. Answer out loud, then reveal.

1. State the Lagrange condition for an equality constraint.

∇f = λ∇h at the optimum, with h = 0.

2. What does the multiplier measure?

The shadow price: the sensitivity of the optimal objective to the constraint.

3. List the four KKT conditions.

Stationarity, primal feasibility, dual feasibility (μ ≥ 0), complementary slackness.

4. What is complementary slackness?

μg = 0: each inequality is active or inactive, not both.

5. When is an inequality constraint active?

When g = 0 at the optimum, with a positive multiplier.

TodayFinish this quiz and Levels 1 and 2 of the ladder.

+1 dayRe-solve the equality and inequality examples from a blank page.

+3 daysApply KKT to three new constrained problems.

+7 daysSpecialise to linear objectives and constraints, Module 7.

+30 daysReuse multipliers as shadow prices in a real design.

Textbook mapping

This module follows Nocedal and Wright, Numerical Optimization, 2nd edition. Use these references to read further.

Topic in this module	Where to read more
Lagrange multipliers and equality constraints	Nocedal & Wright, Chapter 12
The KKT conditions	Nocedal & Wright, Chapter 12
Complementary slackness and sensitivity	Nocedal & Wright, Chapter 12

Chapter numbers refer to the 2nd edition. The constrained-optimization theory is standard, so any recent edition will align closely.