Stochastic Optimal Control
Main.StochasticOptimalControl History
Show minor edits - Show changes to output
Changed line 55 from:
plt.plot(m.time,p.value,'b-',LineWidth=2)
to:
plt.plot(m.time,p.value,'b-',lw=2)
Changed line 58 from:
plt.plot([0,m.time[-1]],[40,40],'k-',LineWidth=3)
to:
plt.plot([0,m.time[-1]],[40,40],'k-',lw=3)
Changed line 60 from:
plt.plot(m.time,v[i].value,':',LineWidth=2)
to:
plt.plot(m.time,v[i].value,':',lw=2)
Changed line 124 from:
plt.plot(m.time,p.value,'b-',LineWidth=2)
to:
plt.plot(m.time,p.value,'b-',lw=2)
Changed line 130 from:
plt.plot(m.time,v[i].value,':',LineWidth=2)
to:
plt.plot(m.time,v[i].value,':',lw=2)
Changed lines 13-14 from:
where ''v'' is the controlled variable (CV) with target value of 40.0 and ''p'' is the manipulated variable (MV) that can be adjusted between 0 and 100. This is a simple first-order dynamic model that relates the adjustable input ''p'' (MV) to the target output ''v'' (CV). The rate of change of the MV is limited to 40 every second. Determine the stochastic optimal control solution by creating 10 instances of the model that randomly determine the value of ''K''. Optimize this set of models to achieve the target CV value of 40, starting from an initial condition of 0.
to:
where ''v'' is the controlled variable (CV) with target value of 40.0 and ''p'' is the manipulated variable (MV) that can be adjusted between 0 and 100. This is a simple first-order dynamic model that relates the adjustable input ''p'' (MV) to the target output ''v'' (CV). The rate of change of the MV is limited to 40 every seconds. Determine the stochastic optimal control solution by creating 10 instances of the model that randomly determine the value of ''K''. Optimize this set of models to achieve the target CV value of 40, starting from an initial condition of 0.
Changed line 73 from:
where ''v'' is the controlled variable (CV) with target range between 38.0 and 40.0, ''p'' is the manipulated variable (MV) that can be adjusted between 0 and 100, and {`\theta`} is the delay. This is a simple first-order dynamic model that relates the adjustable input ''p'' (MV) to the target output ''v'' (CV). The rate of change of the MV is limited to 40 every second. Determine the stochastic optimal control solution by creating 6 instances of the model that randomly determine the delay. Optimize this set of models to achieve the target CV value of between 38 and 40, starting from an initial condition of 0.
to:
where ''v'' is the controlled variable (CV) with target range between 38.0 and 40.0, ''p'' is the manipulated variable (MV) that can be adjusted between 0 and 100, and {`\theta`} is the delay. This is a simple first-order dynamic model that relates the adjustable input ''p'' (MV) to the target output ''v'' (CV). The rate of change of the MV is limited to 40 every seconds. Determine the stochastic optimal control solution by creating 6 instances of the model that randomly determine the delay. Optimize this set of models to achieve the target CV value of between 38 and 40, starting from an initial condition of 0.
Changed lines 15-16 from:
!!!! Solution
to:
!!!! Solution 1
Changed line 75 from:
!!!! Solution
to:
!!!! Solution 2
Changed lines 13-14 from:
where ''v'' is the controlled variable (CV) with target value of 40.0, ''p'' is the manipulated variable (MV) that can be adjusted between 0 and 100. This is a simple first-order dynamic model that relates the adjustable input ''p'' (MV) to the target output ''v'' (CV). The rate of change of the MV is limited to 40 every second. Determine the stochastic optimal control solution by creating 10 instances of the model that randomly determine the value of ''K''. Optimize this set of models to achieve the target CV value of 40, starting from an initial condition of 0.
to:
where ''v'' is the controlled variable (CV) with target value of 40.0 and ''p'' is the manipulated variable (MV) that can be adjusted between 0 and 100. This is a simple first-order dynamic model that relates the adjustable input ''p'' (MV) to the target output ''v'' (CV). The rate of change of the MV is limited to 40 every second. Determine the stochastic optimal control solution by creating 10 instances of the model that randomly determine the value of ''K''. Optimize this set of models to achieve the target CV value of 40, starting from an initial condition of 0.
Changed line 73 from:
where ''v'' is the controlled variable (CV) with target range between 38.0 and 40.0, ''p'' is the manipulated variable (MV) that can be adjusted between 0 and 100. This is a simple first-order dynamic model that relates the adjustable input ''p'' (MV) to the target output ''v'' (CV). The rate of change of the MV is limited to 40 every second. Determine the stochastic optimal control solution by creating 6 instances of the model that randomly determine the delay. Optimize this set of models to achieve the target CV value of between 38 and 40, starting from an initial condition of 0.
to:
where ''v'' is the controlled variable (CV) with target range between 38.0 and 40.0, ''p'' is the manipulated variable (MV) that can be adjusted between 0 and 100, and {`\theta`} is the delay. This is a simple first-order dynamic model that relates the adjustable input ''p'' (MV) to the target output ''v'' (CV). The rate of change of the MV is limited to 40 every second. Determine the stochastic optimal control solution by creating 6 instances of the model that randomly determine the delay. Optimize this set of models to achieve the target CV value of between 38 and 40, starting from an initial condition of 0.
Changed lines 11-12 from:
{`10 \frac{dv}{dt} = -v + K p`}
to:
{$10 \frac{dv}{dt} = -v + K p$}
Changed line 71 from:
{`2 \frac{dv}{dt} = -v + p\left(t-\theta\right)`}
to:
{$2 \frac{dv}{dt} = -v + p (t-\theta)$}
Changed lines 7-8 from:
!!!! Exercise
to:
!!!! Exercise 1
Added lines 66-136:
!!!! Exercise 2
Consider a system that has an uncertain delay that is uniformly distributed between 1.0 and 5.0. The process model is
{`2 \frac{dv}{dt} = -v + p\left(t-\theta\right)`}
where ''v'' is the controlled variable (CV) with target range between 38.0 and 40.0, ''p'' is the manipulated variable (MV) that can be adjusted between 0 and 100. This is a simple first-order dynamic model that relates the adjustable input ''p'' (MV) to the target output ''v'' (CV). The rate of change of the MV is limited to 40 every second. Determine the stochastic optimal control solution by creating 6 instances of the model that randomly determine the delay. Optimize this set of models to achieve the target CV value of between 38 and 40, starting from an initial condition of 0.
!!!! Solution
%width=550px%Attach:optimal_control_delay.png
(:toggle hide gekko2 button show="Show Python GEKKO Source":)
(:div id=gekko2:)
(:source lang=python:)
import numpy as np
from gekko import GEKKO
import matplotlib.pyplot as plt
# uncertain delay instances
n = 6
m = GEKKO()
m.time = np.linspace(0,20,41)
# manipulated variable
p = m.MV(value=0, lb=0, ub=100)
p.STATUS = 1
p.DCOST = 0.1
p.DMAX = 20
# controlled variable
v = m.Array(m.CV,n)
d = m.Array(m.Var,n)
for i in range(n):
delay = np.random.randint(1,11)
v[i].STATUS = 1
v[i].SPHI = 40
v[i].SPLO = 38
v[i].TAU = 5
v[i].TR_INIT = 1
m.delay(p,d[i],delay)
m.Equation(2*v[i].dt()==-v[i]+d[i])
# solve optimal control problem
m.options.IMODE = 6
m.options.CV_TYPE = 1
m.solve()
# get additional solution information
import json
with open(m.path+'//results.json') as f:
results = json.load(f)
# plot results
plt.figure()
plt.subplot(2,1,1)
plt.plot(m.time,p.value,'b-',LineWidth=2)
plt.ylabel('MV')
plt.subplot(2,1,2)
plt.plot(m.time,results['v1.tr_hi'],'k--')
plt.plot(m.time,results['v1.tr_lo'],'k--')
for i in range(n):
plt.plot(m.time,v[i].value,':',LineWidth=2)
plt.ylabel('CV')
plt.xlabel('Time')
plt.show()
(:sourceend:)
(:divend:)
Changed line 9 from:
Consider a system that has an uncertain parameter ''K'' that is uniformly distributed between 10.0 and 11.0. The process model is
to:
Consider a system that has an uncertain parameter ''K'' that is uniformly distributed between 1.0 and 2.0. The process model is
Changed line 13 from:
where ''v'' is the controlled variable (CV) with target value of 40.0, ''p'' is the manipulated variable (MV) that can be adjusted between 0 and 100. This is a simple first-order dynamic model that relates the adjustable input ''p'' (MV) to the target output ''v'' (CV). The rate of change of the MV is limited to 40 every second. Determine the stochastic optimal control solution by creating 10 instances of the model that randomly determine the value of ''K''. Optimize this set of models to achieve the target CV value of 40.
to:
where ''v'' is the controlled variable (CV) with target value of 40.0, ''p'' is the manipulated variable (MV) that can be adjusted between 0 and 100. This is a simple first-order dynamic model that relates the adjustable input ''p'' (MV) to the target output ''v'' (CV). The rate of change of the MV is limited to 40 every second. Determine the stochastic optimal control solution by creating 10 instances of the model that randomly determine the value of ''K''. Optimize this set of models to achieve the target CV value of 40, starting from an initial condition of 0.
Changed line 11 from:
{`10 \frac{dv}{dt} = -v + K \, p`}
to:
{`10 \frac{dv}{dt} = -v + K p`}
Changed line 17 from:
Attach:stochastic_optimal_control.png
to:
%width=550px%Attach:stochastic_optimal_control.png
Added lines 1-65:
(:title Stochastic Optimal Control:)
(:keywords multiple objectives, Python, nonlinear control, model predictive control:)
(:description Stochastic optimal control is a simultaneous optimization of a distribution of process parameters that are sampled from a set of possible process mathematical descriptions.:)
The underlying model or process parameters that describe a system are rarely known exactly. Optimizing a system with an inaccurate model leads to a sub-optimal solution. If the process model includes uncertain parameters that are described by a distribution function or other set of possible process values, this information can be used directly in the optimal control problem to optimize under uncertainty.
!!!! Exercise
Consider a system that has an uncertain parameter ''K'' that is uniformly distributed between 10.0 and 11.0. The process model is
{`10 \frac{dv}{dt} = -v + K \, p`}
where ''v'' is the controlled variable (CV) with target value of 40.0, ''p'' is the manipulated variable (MV) that can be adjusted between 0 and 100. This is a simple first-order dynamic model that relates the adjustable input ''p'' (MV) to the target output ''v'' (CV). The rate of change of the MV is limited to 40 every second. Determine the stochastic optimal control solution by creating 10 instances of the model that randomly determine the value of ''K''. Optimize this set of models to achieve the target CV value of 40.
!!!! Solution
Attach:stochastic_optimal_control.png
(:toggle hide gekko button show="Show Python GEKKO Source":)
(:div id=gekko:)
(:source lang=python:)
import numpy as np
from gekko import GEKKO
import matplotlib.pyplot as plt
# uncertain parameter
n = 10
K = np.random.rand(n)+1.0
m = GEKKO()
m.time = np.linspace(0,20,41)
# manipulated variable
p = m.MV(value=0, lb=0, ub=100)
p.STATUS = 1
p.DCOST = 0.1
p.DMAX = 20
# controlled variable
v = m.Array(m.CV,n)
for i in range(n):
v[i].STATUS = 1
v[i].SP = 40
v[i].TAU = 5
m.Equation(10*v[i].dt() == -v[i] + K[i]*p)
# solve optimal control problem
m.options.IMODE = 6
m.options.CV_TYPE = 2
m.solve()
# plot results
plt.figure()
plt.subplot(2,1,1)
plt.plot(m.time,p.value,'b-',LineWidth=2)
plt.ylabel('MV')
plt.subplot(2,1,2)
plt.plot([0,m.time[-1]],[40,40],'k-',LineWidth=3)
for i in range(n):
plt.plot(m.time,v[i].value,':',LineWidth=2)
plt.ylabel('CV')
plt.xlabel('Time')
plt.show()
(:sourceend:)
(:divend:)
(:keywords multiple objectives, Python, nonlinear control, model predictive control:)
(:description Stochastic optimal control is a simultaneous optimization of a distribution of process parameters that are sampled from a set of possible process mathematical descriptions.:)
The underlying model or process parameters that describe a system are rarely known exactly. Optimizing a system with an inaccurate model leads to a sub-optimal solution. If the process model includes uncertain parameters that are described by a distribution function or other set of possible process values, this information can be used directly in the optimal control problem to optimize under uncertainty.
!!!! Exercise
Consider a system that has an uncertain parameter ''K'' that is uniformly distributed between 10.0 and 11.0. The process model is
{`10 \frac{dv}{dt} = -v + K \, p`}
where ''v'' is the controlled variable (CV) with target value of 40.0, ''p'' is the manipulated variable (MV) that can be adjusted between 0 and 100. This is a simple first-order dynamic model that relates the adjustable input ''p'' (MV) to the target output ''v'' (CV). The rate of change of the MV is limited to 40 every second. Determine the stochastic optimal control solution by creating 10 instances of the model that randomly determine the value of ''K''. Optimize this set of models to achieve the target CV value of 40.
!!!! Solution
Attach:stochastic_optimal_control.png
(:toggle hide gekko button show="Show Python GEKKO Source":)
(:div id=gekko:)
(:source lang=python:)
import numpy as np
from gekko import GEKKO
import matplotlib.pyplot as plt
# uncertain parameter
n = 10
K = np.random.rand(n)+1.0
m = GEKKO()
m.time = np.linspace(0,20,41)
# manipulated variable
p = m.MV(value=0, lb=0, ub=100)
p.STATUS = 1
p.DCOST = 0.1
p.DMAX = 20
# controlled variable
v = m.Array(m.CV,n)
for i in range(n):
v[i].STATUS = 1
v[i].SP = 40
v[i].TAU = 5
m.Equation(10*v[i].dt() == -v[i] + K[i]*p)
# solve optimal control problem
m.options.IMODE = 6
m.options.CV_TYPE = 2
m.solve()
# plot results
plt.figure()
plt.subplot(2,1,1)
plt.plot(m.time,p.value,'b-',LineWidth=2)
plt.ylabel('MV')
plt.subplot(2,1,2)
plt.plot([0,m.time[-1]],[40,40],'k-',LineWidth=3)
for i in range(n):
plt.plot(m.time,v[i].value,':',LineWidth=2)
plt.ylabel('CV')
plt.xlabel('Time')
plt.show()
(:sourceend:)
(:divend:)