Chapter 3: Unconstrained Optimization

In this section we will examine some theory for the optimization of unconstrained functions. We will assume all functions are continuous and differentiable. Although most engineering problems are constrained, much of constrained optimization theory is built upon the concepts and theory presented in this section.

Discussion Questions

Bring answers to class or work them with an AI tutor - ask it to grade your answer and argue the other side before it explains.

1. A classmate finds a point where the gradient is zero and declares victory. Give three different things that point could be, and state what the Hessian must satisfy for it to be a strict local minimum. What extra property of the problem would make it the GLOBAL minimum?
2. Line search and trust region are two philosophies for turning a search direction into a step. Contrast them: which commits to a direction first and which to a step size, and why do trust regions cope better when the local quadratic model is bad?
3. Propose a convergence test better than "the objective stopped changing." Why is a gradient-norm tolerance still not the whole story on a badly scaled problem, and what do practical codes (like IPOPT) check instead?
4. Steepest descent moves in the "best" local direction yet can be spectacularly slow. Explain the zigzag on elongated contours, and why rescaling the variables - not changing the algorithm - can fix it. What does this say about diagnosing "slow solver" complaints in practice?