Quiz on Nonlinear Programming

1. Which of the following is not one of the Karush Kuhn Tucker conditions?

A. Solution is feasible
Incorrect. KKT #1: all equations are satisfied
B. Improved objective in any feasible direction
Correct. KKT #2: objective is worse in any feasible direction, not better.
C. Lagrange multiplier or the constraint must be equal to zero
Incorrect. KKT #3: complementarity slackness
D. Lagrange multipliers must be greater than or equal to zero
Incorrect. KKT #4: Zero or positive Lagrange multipliers. An alternative definition of Lagrange multipliers is that they must be less than or equal to zero by switching a negative sign in the definition of the Lagrangian.

2. Solve the optimization problem.

$$\min x_1 x_4 \left(x_1 + x_2 + x_3\right) + x_3$$ $$\mathrm{s.t.} \quad x_1 x_2 x_3 x_4 \ge 25$$ $$x_1^2 + x_2^2 + x_3^2 + x_4^2 = 40$$ $$1.4\le x_1 \le 5$$ $$1\le x_2, x_3, x_4 \le 5$$ $$x_0 = (1,5,5,1)$$

Change the code so that `x_1` has a lower bound of 1.4 and solve with Scipy.optimize.minimize.

from scipy.optimize import minimize
def objective(x):
    return x[0]*x[3]*(x[0]+x[1]+x[2])+x[2]
def constraint1(x):
    return x[0]*x[1]*x[2]*x[3]-25.0
def constraint2(x):
    return 40.0 - sum([xi**2 for xi in x])
x0 = [1,5,5,1] # initial guesses

print('Initial SSE Objective: ' + str(objective(x0)))

# optimize
b = (1.0,5.0)
bnds = (b, b, b, b)
con1 = {'type': 'ineq', 'fun': constraint1}
con2 = {'type': 'eq', 'fun': constraint2}
cons = ([con1,con2])
solution = minimize(objective,x0,method='SLSQP',\
                    bounds=bnds,constraints=cons)
x = solution.x
print('Final SSE Objective: ' + str(objective(x)))
print('x1 = ' + str(x[0])); print('x2 = ' + str(x[1]))
print('x3 = ' + str(x[2])); print('x4 = ' + str(x[3]))
[$[Get Code]]

What is the value of the objective function when solving the problem with `x_1>=1.4`?

A. 17.59
Correct. The objective function always gets worse or stays the same with added constraints. The original objective function is 17.01. The added constraint makes the solution worse because the objective is to minimize.
from scipy.optimize import minimize
def objective(x):
    return x[0]*x[3]*(x[0]+x[1]+x[2])+x[2]
def constraint1(x):
    return x[0]*x[1]*x[2]*x[3]-25.0
def constraint2(x):
    return 40.0 - sum([xi**2 for xi in x])
x0 = [1,5,5,1] # initial guesses

print('Initial SSE Objective: ' + str(objective(x0)))

# optimize
b = (1.0,5.0)
bnds = ((1.4,5.0), b, b, b)
con1 = {'type': 'ineq', 'fun': constraint1}
con2 = {'type': 'eq', 'fun': constraint2}
cons = ([con1,con2])
solution = minimize(objective,x0,method='SLSQP',\
                    bounds=bnds,constraints=cons)
x = solution.x
print('Final SSE Objective: ' + str(objective(x)))
print('x1 = ' + str(x[0])); print('x2 = ' + str(x[1]))
print('x3 = ' + str(x[2])); print('x4 = ' + str(x[3]))
[$[Get Code]]
B. 22.39
Incorrect. Change only the bounds on `x_1`, not on all the variables
C. 16
Incorrect. This is the objective with the guess value. It is not a solution because it is infeasible because the second constraint is not satisfied
D. 17.01
Incorrect. This is the original objective without changing the bound on `x_1`.

3. Solve the optimization problem.

$$\min x_1 x_4 \left(x_1 + x_2 + x_3\right) + x_3$$ $$\mathrm{s.t.} \quad x_1 x_2 x_3 x_4 \ge 25$$ $$x_1^2 + x_2^2 + x_3^2 + x_4^2 = 40$$ $$x_1+x_4\ge3$$ $$1\le x_1, x_2, x_3, x_4 \le 5$$ $$x_0 = (1,5,5,1)$$

Add a constraint `x_1 + x_4 \geq 3` to the Python Gekko code.

from gekko import GEKKO    
m = GEKKO(remote=False)

# create variables
x1,x2,x3,x4 = m.Array(m.Var,4,lb=1,ub=5)

# initial guess values
x1.value = 1; x2.value = 5; x3.value = 5; x4.value = 1

m.Equation(x1*x2*x3*x4>=25)
m.Equation(x1**2+x2**2+x3**2+x4**2==40)
m.Minimize(x1*x4*(x1+x2+x3)+x3)
m.solve(disp=False)

print('Results')
print('x1: ' + str(x1.value[0])); print('x2: ' + str(x2.value[0]))
print('x3: ' + str(x3.value[0])); print('x4: ' + str(x4.value[0]))
[$[Get Code]]

Solve the optimization problem with the added constraint. What is the value of `x_3`?

A. 3.16
Correct.
from gekko import GEKKO    
m = GEKKO(remote=False)

# create variables
x1,x2,x3,x4 = m.Array(m.Var,4,lb=1,ub=5)

# initial guess values
x1.value = 1; x2.value = 5; x3.value = 5; x4.value = 1

m.Equation(x1*x2*x3*x4>=25)
m.Equation(x1**2+x2**2+x3**2+x4**2==40)
m.Equation(x1+x4>=3)
m.Minimize(x1*x4*(x1+x2+x3)+x3)
m.solve(disp=False)

print('Results')
print('x1: ' + str(x1.value[0])); print('x2: ' + str(x2.value[0]))
print('x3: ' + str(x3.value[0])); print('x4: ' + str(x4.value[0]))
[$[Get Code]]
B. 3.82
Incorrect. Add the new constraint with m.Equation(x1+x4>=3)
C. 5.69
Incorrect. This solution cannot be correct because it is above the upper bound of 5.
D. 5.00
Incorrect. This is the value of `x_2`