Inverted Pendulum Optimal Control

Main.InvertedPendulum History

Hide minor edits - Show changes to output

March 01, 2023, at 06:02 PM by 10.35.117.248 -
Changed lines 21-22 from:
Attach:inverted_pendulum.gif
to:
%width=400px%Attach:inverted_pendulum.gif
Changed line 198 from:
Attach:inverted_pendulum_wgts.gif
to:
%width=400px%Attach:inverted_pendulum_wgts.gif
March 01, 2023, at 06:01 PM by 10.35.117.248 -
Changed lines 17-18 from:
where ''u'' is the force applied to the cart, {`\epsilon`} is ''m'_2_'/(m'_1_'+m'_2_')'', y is the position of the cart, ''v'' is the velocity of the cart, {`\theta`} is the angle of the pendulum relative to the cart, ''m'_1_'''=10, ''m'_2_'''=1, and ''q'' is the rate of angle change. Tune the controller to minimize the use of force applied to the cart either in the forward or reverse direction (i.e. minimize fuel consumed to perform the maneuver). Explain the tuning and the optimal solution with appropriate plots that demonstrate that the solution is optimal. The [[Main/ControlTypes|non-inverted pendulum problem]] 
 and a [[Main/DoubleInvertedPendulum|nonlinear double inverted pendulum]] are additional examples.
to:
where ''u'' is the force applied to the cart, {`\epsilon`} is ''m'_2_'/(m'_1_'+m'_2_')'', y is the position of the cart, ''v'' is the velocity of the cart, {`\theta`} is the angle of the pendulum relative to the cart, ''m'_1_'''=10, ''m'_2_'''=1, and ''q'' is the rate of angle change. Tune the controller to minimize the use of force applied to the cart either in the forward or reverse direction (i.e. minimize fuel consumed to perform the maneuver). Explain the tuning and the optimal solution with appropriate plots that demonstrate that the solution is optimal. The [[Main/ControlTypes|non-inverted pendulum problem]] and a [[Main/DoubleInvertedPendulum|nonlinear double inverted pendulum]] are additional examples.
March 01, 2023, at 06:01 PM by 10.35.117.248 -
Changed lines 17-18 from:
where ''u'' is the force applied to the cart, {`\epsilon`} is ''m'_2_'/(m'_1_'+m'_2_')'', y is the position of the cart, ''v'' is the velocity of the cart, {`\theta`} is the angle of the pendulum relative to the cart, ''m'_1_'''=10, ''m'_2_'''=1, and ''q'' is the rate of angle change. Tune the controller to minimize the use of force applied to the cart either in the forward or reverse direction (i.e. minimize fuel consumed to perform the maneuver). Explain the tuning and the optimal solution with appropriate plots that demonstrate that the solution is optimal. A similar but [[Main/ControlTypes|non-inverted pendulum problem]] is posted on the course web-site as an example in the introductory section for model predictive control.
to:
where ''u'' is the force applied to the cart, {`\epsilon`} is ''m'_2_'/(m'_1_'+m'_2_')'', y is the position of the cart, ''v'' is the velocity of the cart, {`\theta`} is the angle of the pendulum relative to the cart, ''m'_1_'''=10, ''m'_2_'''=1, and ''q'' is the rate of angle change. Tune the controller to minimize the use of force applied to the cart either in the forward or reverse direction (i.e. minimize fuel consumed to perform the maneuver). Explain the tuning and the optimal solution with appropriate plots that demonstrate that the solution is optimal. The [[Main/ControlTypes|non-inverted pendulum problem]]
 and a [[Main/DoubleInvertedPendulum|nonlinear double inverted pendulum]] are additional examples
.
January 06, 2022, at 05:21 PM by 10.35.117.248 -
Added lines 405-406:

See additional information on [[https://apmonitor.com/pds/index.php/Main/HandTracking|Hand Tracking]] to control the cart position. Reinforcement Learning or Model Predictive Control can also be used to control the cart position (see Additional Reading).
January 04, 2022, at 07:04 PM by 10.35.117.248 -
Changed lines 386-387 from:
OpenAI has a similar benchmark problem for testing Reinforcement Learning. Similar to the problem above, a pole is attached by a frictionless joint and cart that moves along a track. The cart is controlled by applying a force of -1 to +1. The pendulum starts upright and a reward of +1 is provided for every timestep that the pole remains upright. The gym returns ''done=True'' when the pole is more than 15° from vertical or the cart is more than 2.4 units from the starting location.
to:
OpenAI has a similar benchmark problem for testing Reinforcement Learning. Similar to the problem above, a pole is attached by a frictionless joint and cart that moves along a track. The cart is controlled by applying a force to the left (action=0) or right (action=1). The pendulum starts upright and a reward of +1 is provided for every timestep that the pole remains upright. The gym returns ''done=True'' when the pole is more than 15° from vertical or the cart is more than 2.4 units from the starting location.
Changed line 397 from:
   #  +1 to -1 force to the cart
to:
   #  Force to the cart with actions: 0=left, 1=right
Added lines 387-388:

%width=550px%Attach:cartpole.gif
Changed lines 384-390 from:
!!!! Additional Reading

* Bryson, A.E., Dynamic Optimization, Addison-Wesley, 1999.
* Balawejder, M., Solving Open AI’s CartPole using Reinforcement Learning
, March 31, 2021. [[https://medium.com/analytics-vidhya/q-learning-is-the-most-basic-form-of-reinforcement-learning-which-doesnt-take-advantage-of-any-8944e02570c5|Article]].
* Barto, A.G., Neuronlike adaptive elements
that can solve difficult learning control problems, IEEE Transactions on Systems, Man, and Cybernetics, Volume: SMC-13, Issue: 5, 1983. [[https://ieeexplore.ieee.org/abstract/document/6313077|Article]]
* OpenAI, [[https://gym.openai.com/envs/CartPole-v0/|Cart Pole v0 Gym]] | [[https://gym.openai.com/docs/|Documentation]] | [[https://github.com/pytorch/examples/tree/master/reinforcement_learning|PyTorch RL Solution]] | [[https://www.tensorflow.org/tutorials/reinforcement_learning/actor_critic|TensorFlow RL Solution]] | [[https://keras.io/examples/rl/actor_critic_cartpole/|Keras RL Solution]]

to:
!!!! OpenAI Cart Pole

OpenAI has a similar benchmark problem for testing Reinforcement Learning. Similar to the problem above
, a pole is attached by a frictionless joint and cart that moves along a track. The cart is controlled by applying a force of -1 to +1. The pendulum starts upright and a reward of +1 is provided for every timestep that the pole remains upright. The gym returns ''done=True'' when the pole is more than 15° from vertical or the cart is more than 2.4 units from the starting location.
Added lines 394-399:
   # Input:
    #  +1 to -1 force to the cart
    # Returns:
    #  obs = cart position, cart velocity, pole angle, rot rate
    #  reward = +1 for every timestep
    #  done = True when abs(angle)>15 or abs(cart pos)>2.4
Added lines 403-409:

!!!! Additional Reading

* Bryson, A.E., Dynamic Optimization, Addison-Wesley, 1999.
* Balawejder, M., Solving Open AI’s CartPole using Reinforcement Learning, March 31, 2021. [[https://medium.com/analytics-vidhya/q-learning-is-the-most-basic-form-of-reinforcement-learning-which-doesnt-take-advantage-of-any-8944e02570c5|Article]].
* Barto, A.G., Neuronlike adaptive elements that can solve difficult learning control problems, IEEE Transactions on Systems, Man, and Cybernetics, Volume: SMC-13, Issue: 5, 1983. [[https://ieeexplore.ieee.org/abstract/document/6313077|Article]]
* OpenAI, [[https://gym.openai.com/envs/CartPole-v0/|Cart Pole v0 Gym]] | [[https://gym.openai.com/docs/|Documentation]] | [[https://github.com/pytorch/examples/tree/master/reinforcement_learning|PyTorch RL Solution]] | [[https://www.tensorflow.org/tutorials/reinforcement_learning/actor_critic|TensorFlow RL Solution]] | [[https://keras.io/examples/rl/actor_critic_cartpole/|Keras RL Solution]]
Changed line 397 from:
   r = env.step(env.action_space.sample())
to:
   obs,reward,done,info = env.step(env.action_space.sample())
Added lines 390-399:

(:source lang=python:)
import gym
env = gym.make('CartPole-v0')
env.reset()
for i in range(100):
    env.render()
    r = env.step(env.action_space.sample())
env.close()
(:sourceend:)
Changed line 389 from:
* OpenAI, Cart Pole v0, [[https://gym.openai.com/envs/CartPole-v0/|Gym]] | [[https://gym.openai.com/docs/|Documentation]] | [[https://github.com/pytorch/examples/tree/master/reinforcement_learning|PyTorch RL Solution]] | [[https://www.tensorflow.org/tutorials/reinforcement_learning/actor_critic|TensorFlow RL Solution]] | [[https://keras.io/examples/rl/actor_critic_cartpole/|Keras RL Solution]]
to:
* OpenAI, [[https://gym.openai.com/envs/CartPole-v0/|Cart Pole v0 Gym]] | [[https://gym.openai.com/docs/|Documentation]] | [[https://github.com/pytorch/examples/tree/master/reinforcement_learning|PyTorch RL Solution]] | [[https://www.tensorflow.org/tutorials/reinforcement_learning/actor_critic|TensorFlow RL Solution]] | [[https://keras.io/examples/rl/actor_critic_cartpole/|Keras RL Solution]]
Changed line 389 from:
* OpenAI, Cart Pole v0, [[https://gym.openai.com/envs/CartPole-v0/|Gym]] | [[https://gym.openai.com/docs/|Documentation]] | [[https://github.com/pytorch/examples/tree/master/reinforcement_learning|PyTorch RL Solution]] | [[https://www.tensorflow.org/tutorials/reinforcement_learning/actor_critic|TensorFlow RL Solution]]
to:
* OpenAI, Cart Pole v0, [[https://gym.openai.com/envs/CartPole-v0/|Gym]] | [[https://gym.openai.com/docs/|Documentation]] | [[https://github.com/pytorch/examples/tree/master/reinforcement_learning|PyTorch RL Solution]] | [[https://www.tensorflow.org/tutorials/reinforcement_learning/actor_critic|TensorFlow RL Solution]] | [[https://keras.io/examples/rl/actor_critic_cartpole/|Keras RL Solution]]
Changed line 389 from:
* OpenAI, Cart Pole v0, [[https://gym.openai.com/envs/CartPole-v0/|Gym]] | [[https://gym.openai.com/docs/|Documentation]]
to:
* OpenAI, Cart Pole v0, [[https://gym.openai.com/envs/CartPole-v0/|Gym]] | [[https://gym.openai.com/docs/|Documentation]] | [[https://github.com/pytorch/examples/tree/master/reinforcement_learning|PyTorch RL Solution]] | [[https://www.tensorflow.org/tutorials/reinforcement_learning/actor_critic|TensorFlow RL Solution]]
Changed lines 384-385 from:
!!!! Reference
to:
!!!! Additional Reading
Changed lines 387-389 from:
to:
* Balawejder, M., Solving Open AI’s CartPole using Reinforcement Learning, March 31, 2021. [[https://medium.com/analytics-vidhya/q-learning-is-the-most-basic-form-of-reinforcement-learning-which-doesnt-take-advantage-of-any-8944e02570c5|Article]].
* Barto, A.G., Neuronlike adaptive elements that can solve difficult learning control problems, IEEE Transactions on Systems, Man, and Cybernetics, Volume: SMC-13, Issue: 5, 1983. [[https://ieeexplore.ieee.org/abstract/document/6313077|Article]]
* OpenAI, Cart Pole v0, [[https://gym.openai.com/envs/CartPole-v0/|Gym]] | [[https://gym.openai.com/docs/|Documentation]]
August 05, 2019, at 08:48 PM by 184.91.225.255 -
Changed lines 80-83 from:
m.fix(ya,end_loc,0.0)
m.fix(va,end_loc,0.0)
m.fix(theta_a,end_loc,0.0)
m.fix(qa,end_loc,0.0)
to:
m.fix(ya,pos=end_loc,val=0.0)
m.fix(va,pos=end_loc,val=0.0)
m.fix(theta_a,pos=end_loc,val=0.0)
m.fix(qa,pos=end_loc,val=0.0)
Changed lines 255-259 from:
m.fix(ya,i,0.0)
m.fix(va,i,0.0)
m.fix(theta_a,i,0.0)
m.fix(qa,i,0.0)
to:
m.fix(ya,pos=i,val=0.0)
m.fix(va,pos=i,val=0.0)
m.fix(theta_a,pos=i,val=0.0)
m.fix(qa,pos=i,val=0.0)
Changed lines 261-265 from:
m.fix(ya,i,-1.0)
m.fix(va,i,0.0)
m.fix(theta_a,i,0.0)
m.fix(qa,i,0.0)
to:
m.fix(ya,pos=i,val=-1.0)
m.fix(va,pos=i,val=0.0)
m.fix(theta_a,pos=i,val=0.0)
m.fix(qa,pos=i,val=0.0)
Changed lines 267-271 from:
m.fix(ya,i,0.0)
m.fix(va,i,0.0)
m.fix(theta_a,i,0.0)
m.fix(qa,i,0.0)
to:
m.fix(ya,pos=i,val=0.0)
m.fix(va,pos=i,val=0.0)
m.fix(theta_a,pos=i,val=0.0)
m.fix(qa,pos=i,val=0.0)
Changed lines 273-276 from:
m.fix(ya,i,-1.0)
m.fix(va,i,0.0)
m.fix(theta_a,i,0.0)
m.fix(qa,i,0.0)
to:
m.fix(ya,pos=i,val=-1.0)
m.fix(va,pos=i,val=0.0)
m.fix(theta_a,pos=i,val=0.0)
m.fix(qa,pos=i,val=0.0)
March 22, 2018, at 07:36 PM by 45.56.3.173 -
Changed lines 19-20 from:
!!!! Solution
to:
!!!! Python (GEKKO) Solution
Added lines 188-189:
!!!! APM MATLAB and APM Python Solution
Added lines 195-382:

!!!! Response with Different Weights

Attach:inverted_pendulum_wgts.gif

(:toggle hide gekko_animate2 button show="Show GEKKO (Python) Code":)
(:div id=gekko_animate2:)
(:source lang=python:)
import matplotlib.animation as animation
import numpy as np
from gekko import GEKKO

#Defining a model
m = GEKKO()

#Defining the time, we will go beyond the 6.2s
#to check if the objective was achieved
n = 300
tf = 24.0
m.time = np.linspace(0,tf,n)
end_loc1 = int(n*6.0/tf)
end_loc2 = int(n*12.0/tf)
end_loc3 = int(n*18.0/tf)
end_loc4 = n-1 # end

#################################
#Weight of item
m2 = np.ones(n)
m2[0:end_loc1] = 0.1
m2[end_loc1:end_loc2] = 1.0
m2[end_loc2:end_loc3] = 10.0
m2[end_loc3:end_loc4] = 20.0
#################################


#Parameters
m1a = m.Param(value=10)
m2a = m.Param(value=m2)

#MV
ua = m.Var(value=0)

#State Variables
theta_a = m.Var(value=0)
qa = m.Var(value=0)
ya = m.Var(value=-1)
va = m.Var(value=0)

#Intermediates
epsilon = m.Intermediate(m2a/(m1a+m2a))

#Defining the State Space Model
m.Equation(ya.dt() == va)
m.Equation(va.dt() == -epsilon*theta_a + ua)
m.Equation(theta_a.dt() == qa)
m.Equation(qa.dt() == theta_a -ua)

#Definine the Objectives
#Make all the state variables be zero at endpoints
i = end_loc1
m.fix(ya,i,0.0)
m.fix(va,i,0.0)
m.fix(theta_a,i,0.0)
m.fix(qa,i,0.0)

i = end_loc2
m.fix(ya,i,-1.0)
m.fix(va,i,0.0)
m.fix(theta_a,i,0.0)
m.fix(qa,i,0.0)

i = end_loc3
m.fix(ya,i,0.0)
m.fix(va,i,0.0)
m.fix(theta_a,i,0.0)
m.fix(qa,i,0.0)

i = end_loc4
m.fix(ya,i,-1.0)
m.fix(va,i,0.0)
m.fix(theta_a,i,0.0)
m.fix(qa,i,0.0)

#Try to minimize change of MV over all horizon
m.Obj(0.001*ua**2)

m.options.SOLVER = 3
m.options.IMODE = 6 #MPC
m.solve() #(disp=False)

#Plotting the results
import matplotlib.pyplot as plt
plt.figure(figsize=(12,10))

plt.subplot(221)
plt.plot(m.time,ua.value,'m',lw=2)
plt.legend([r'$u$'],loc=1)
plt.ylabel('Force')
plt.xlabel('Time')
plt.xlim(m.time[0],m.time[-1])

plt.subplot(222)
plt.plot(m.time,va.value,'g',lw=2)
plt.legend([r'$v$'],loc=1)
plt.ylabel('Velocity')
plt.xlabel('Time')
plt.xlim(m.time[0],m.time[-1])

plt.subplot(223)
plt.plot(m.time,ya.value,'r',lw=2)
plt.legend([r'$y$'],loc=1)
plt.ylabel('Position')
plt.xlabel('Time')
plt.xlim(m.time[0],m.time[-1])

plt.subplot(224)
plt.plot(m.time,theta_a.value,'y',lw=2)
plt.plot(m.time,qa.value,'c',lw=2)
plt.legend([r'$\theta$',r'$q$'],loc=1)
plt.ylabel('Angle')
plt.xlabel('Time')
plt.xlim(m.time[0],m.time[-1])

plt.rcParams['animation.html'] = 'html5'

x1 = ya.value
y1 = np.zeros(len(m.time))

#suppose that l = 1
x2 = 1*np.sin(theta_a.value)+x1
x2b = 1.05*np.sin(theta_a.value)+x1
y2 = 1*np.cos(theta_a.value)-y1
y2b = 1.05*np.cos(theta_a.value)-y1

fig = plt.figure(figsize=(8,6.4))
ax = fig.add_subplot(111,autoscale_on=False,\
                    xlim=(-1.8,0.8),ylim=(-0.4,1.2))
ax.set_xlabel('position')
ax.get_yaxis().set_visible(False)

crane_rail, = ax.plot([-2.0,1.0],[-0.2,-0.2],'k-',lw=4)
start, = ax.plot([-1,-1],[-1.5,1.5],'k:',lw=2)
objective, = ax.plot([0,0],[-0.5,1.5],'k:',lw=2)
mass1, = ax.plot([],[],linestyle='None',marker='s',\
                markersize=40,markeredgecolor='k',\
                color='orange',markeredgewidth=2)
mass2, = ax.plot([],[],linestyle='None',marker='o',\
                markersize=20,markeredgecolor='k',\
                color='orange',markeredgewidth=2)
line, = ax.plot([],[],'o-',color='orange',lw=4,\
                markersize=6,markeredgecolor='k',\
                markerfacecolor='k')
time_template = 'time = %.1fs'
time_text = ax.text(0.05,0.9,'',transform=ax.transAxes)
wgt_template = 'weight = %.1f'
wgt_text = ax.text(0.75,0.9,'',transform=ax.transAxes)
start_text = ax.text(-1.06,-0.3,'start',ha='right')
end_text = ax.text(0.06,-0.3,'objective',ha='left')

def init():
    mass1.set_data([],[])
    mass2.set_data([],[])
    line.set_data([],[])
    time_text.set_text('')
    wgt_text.set_text('')
    return line, mass1, mass2, time_text, wgt_text

def animate(i):
    mass1.set_data([x1[i]],[y1[i]-0.1])
    mass2.set_data([x2b[i]],[y2b[i]])
    line.set_data([x1[i],x2[i]],[y1[i],y2[i]])
    time_text.set_text(time_template % m.time[i])
    wgt_text.set_text(wgt_template % m2[i])
    return line, mass1, mass2, time_text, wgt_text

ani_a = animation.FuncAnimation(fig, animate, \
        np.arange(1,len(m.time)), \
        interval=40,blit=False,init_func=init)

# requires ffmpeg to save mp4 file
#  available from https://ffmpeg.zeranoe.com/builds/
#  add ffmpeg.exe to path such as C:\ffmpeg\bin\ in
#  environment variables
ani_a.save('Pendulum_Control.mp4',fps=30)

plt.show()
(:sourceend:)
(:divend:)
March 22, 2018, at 06:32 PM by 45.56.3.173 -
Added lines 6-9:

(:html:)
<iframe width="560" height="315" src="https://www.youtube.com/embed/egPRF5PHwOo" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
(:htmlend:)
March 22, 2018, at 12:26 AM by 10.37.35.33 -
Changed lines 17-184 from:
Attach:download.png [[Attach:final_solution2_files.zip|Inverted Pendulum: Solution Files]]
to:
Attach:inverted_pendulum.gif

(:toggle hide gekko_animate button show="Show GEKKO (Python) and Animation Code":)
(:div id=gekko_animate:)
(:source lang=python:)
# Contributed by Everton Colling
import matplotlib.animation as animation
import numpy as np
from gekko import GEKKO

#Defining a model
m = GEKKO()

#################################
#Weight of item
m2 = 1
#################################

#Defining the time, we will go beyond the 6.2s
#to check if the objective was achieved
m.time = np.linspace(0,8,100)
end_loc = int(100.0*6.2/8.0)

#Parameters
m1a = m.Param(value=10)
m2a = m.Param(value=m2)
final = np.zeros(len(m.time))
for i in range(len(m.time)):
    if m.time[i] < 6.2:
        final[i] = 0
    else:
        final[i] = 1
final = m.Param(value=final)

#MV
ua = m.Var(value=0)

#State Variables
theta_a = m.Var(value=0)
qa = m.Var(value=0)
ya = m.Var(value=-1)
va = m.Var(value=0)

#Intermediates
epsilon = m.Intermediate(m2a/(m1a+m2a))

#Defining the State Space Model
m.Equation(ya.dt() == va)
m.Equation(va.dt() == -epsilon*theta_a + ua)
m.Equation(theta_a.dt() == qa)
m.Equation(qa.dt() == theta_a -ua)

#Definine the Objectives
#Make all the state variables be zero at time >= 6.2
m.Obj(final*ya**2)
m.Obj(final*va**2)
m.Obj(final*theta_a**2)
m.Obj(final*qa**2)

m.fix(ya,end_loc,0.0)
m.fix(va,end_loc,0.0)
m.fix(theta_a,end_loc,0.0)
m.fix(qa,end_loc,0.0)
#Try to minimize change of MV over all horizon
m.Obj(0.001*ua**2)

m.options.IMODE = 6 #MPC
m.solve() #(disp=False)

#Plotting the results
import matplotlib.pyplot as plt
plt.figure(figsize=(12,10))

plt.subplot(221)
plt.plot(m.time,ua.value,'m',lw=2)
plt.legend([r'$u$'],loc=1)
plt.ylabel('Force')
plt.xlabel('Time')
plt.xlim(m.time[0],m.time[-1])

plt.subplot(222)
plt.plot(m.time,va.value,'g',lw=2)
plt.legend([r'$v$'],loc=1)
plt.ylabel('Velocity')
plt.xlabel('Time')
plt.xlim(m.time[0],m.time[-1])

plt.subplot(223)
plt.plot(m.time,ya.value,'r',lw=2)
plt.legend([r'$y$'],loc=1)
plt.ylabel('Position')
plt.xlabel('Time')
plt.xlim(m.time[0],m.time[-1])

plt.subplot(224)
plt.plot(m.time,theta_a.value,'y',lw=2)
plt.plot(m.time,qa.value,'c',lw=2)
plt.legend([r'$\theta$',r'$q$'],loc=1)
plt.ylabel('Angle')
plt.xlabel('Time')
plt.xlim(m.time[0],m.time[-1])

plt.rcParams['animation.html'] = 'html5'

x1 = ya.value
y1 = np.zeros(len(m.time))

#suppose that l = 1
x2 = 1*np.sin(theta_a.value)+x1
x2b = 1.05*np.sin(theta_a.value)+x1
y2 = 1*np.cos(theta_a.value)-y1
y2b = 1.05*np.cos(theta_a.value)-y1

fig = plt.figure(figsize=(8,6.4))
ax = fig.add_subplot(111,autoscale_on=False,\
                    xlim=(-1.5,0.5),ylim=(-0.4,1.2))
ax.set_xlabel('position')
ax.get_yaxis().set_visible(False)

crane_rail, = ax.plot([-1.5,0.5],[-0.2,-0.2],'k-',lw=4)
start, = ax.plot([-1,-1],[-1.5,1.5],'k:',lw=2)
objective, = ax.plot([0,0],[-0.5,1.5],'k:',lw=2)
mass1, = ax.plot([],[],linestyle='None',marker='s',\
                markersize=40,markeredgecolor='k',\
                color='orange',markeredgewidth=2)
mass2, = ax.plot([],[],linestyle='None',marker='o',\
                markersize=20,markeredgecolor='k',\
                color='orange',markeredgewidth=2)
line, = ax.plot([],[],'o-',color='orange',lw=4,\
                markersize=6,markeredgecolor='k',\
                markerfacecolor='k')
time_template = 'time = %.1fs'
time_text = ax.text(0.05,0.9,'',transform=ax.transAxes)
start_text = ax.text(-1.06,-0.3,'start',ha='right')
end_text = ax.text(0.06,-0.3,'objective',ha='left')

def init():
    mass1.set_data([],[])
    mass2.set_data([],[])
    line.set_data([],[])
    time_text.set_text('')
    return line, mass1, mass2, time_text

def animate(i):
    mass1.set_data([x1[i]],[y1[i]-0.1])
    mass2.set_data([x2b[i]],[y2b[i]])
    line.set_data([x1[i],x2[i]],[y1[i],y2[i]])
    time_text.set_text(time_template % m.time[i])
    return line, mass1, mass2, time_text

ani_a = animation.FuncAnimation(fig, animate, \
        np.arange(1,len(m.time)), \
        interval=40,blit=False,init_func=init)

# requires ffmpeg to save mp4 file
#  available from https://ffmpeg.zeranoe.com/builds/
#  add ffmpeg.exe to path such as C:\ffmpeg\bin\ in
#  environment variables

#ani_a.save('Pendulum_Control.mp4',fps=30)

plt.show()
(:sourceend:)

Thanks to [[https://www.linkedin.com/in/everton-colling-0161b047/|Everton Colling]] for the animation code in Python.
(:divend:)

Attach:download.png [[Attach:final_solution2_files.zip|Inverted Pendulum: Solution Files (APM Python and APM MATLAB)]]
March 22, 2018, at 12:18 AM by 10.37.35.33 -
Changed lines 11-35 from:
{$\begin{bmatrix}
\dot y \\
\dot v \\
\dot \theta \\
\dot q
\end{bmatrix}=
\begin{bmatrix}
0 & 1 & 0 & 0\\
0 & 0 & -\epsilon & 0\\
0 & 0 & 0 & 1\\
0 & 0 & 1 & 0
\end{bmatrix} \begin{bmatrix}
y \\
v \\
\theta \\
q
\end{bmatrix} +

\begin{bmatrix}
0 \\
1 \\
0 \\
-1
\end{bmatrix} u
$}
to:
{$\begin{bmatrix} \dot y \\ \dot v \\ \dot \theta \\ \dot q \end{bmatrix}=\begin{bmatrix} 0 & 1 & 0 & 0\\ 0 & 0 & -\epsilon & 0\\ 0 & 0 & 0 & 1\\ 0 & 0 & 1 & 0 \end{bmatrix} \begin{bmatrix} y \\ v \\ \theta \\ q \end{bmatrix} + \begin{bmatrix} 0 \\ 1 \\ 0 \\ -1 \end{bmatrix} u$}
March 22, 2018, at 12:17 AM by 10.37.35.33 -
Changed line 11 from:
\begin{bmatrix}
to:
{$\begin{bmatrix}
Changed lines 16-35 from:
\end{bmatrix}
to:
\end{bmatrix}=
\begin{bmatrix}
0 & 1 & 0 & 0\\
0 & 0 & -\epsilon & 0\\
0 & 0 & 0 & 1\\
0 & 0 & 1 & 0
\end{bmatrix} \begin{bmatrix}
y \\
v \\
\theta \\
q
\end{bmatrix} +

\begin{bmatrix}
0 \\
1 \\
0 \\
-1
\end{bmatrix} u
$
}
March 22, 2018, at 12:13 AM by 10.37.35.33 -
Changed line 7 from:
%width=300px%Attach:inverted_pendulum.png
to:
%width=200px%Attach:inverted_pendulum.png
March 22, 2018, at 12:13 AM by 10.37.35.33 -
Changed lines 5-7 from:
Design a model predictive controller for an inverted pendulum system with an adjustable cart. Demonstrate that the cart can perform a sequence of moves to maneuver from position ''y''=-1.0 to ''y''=0.0 within 6.2 seconds. Verify that ''v'', {`\theta`}, and ''q'' are zero before and after the maneuver. 
to:
Design a model predictive controller for an inverted pendulum system with an adjustable cart. Demonstrate that the cart can perform a sequence of moves to maneuver from position ''y''=-1.0 to ''y''=0.0 within 6.2 seconds. Verify that ''v'', {`\theta`}, and ''q'' are zero before and after the maneuver.

%width=300px%Attach:inverted_pendulum.png
March 22, 2018, at 12:12 AM by 10.37.35.33 -
Changed line 16 from:
where ''u'' is the force applied to the cart, {`\epsilon`} is ''m2/(m1+m2)'', y is the position of the cart, ''v'' is the velocity of the cart, {`\theta`} is the angle of the pendulum relative to the cart, ''m1''=10, ''m2''=1, and ''q'' is the rate of angle change. Tune the controller to minimize the use of force applied to the cart either in the forward or reverse direction (i.e. minimize fuel consumed to perform the maneuver). Explain the tuning and the optimal solution with appropriate plots that demonstrate that the solution is optimal. A similar but [[Main/ControlTypes|non-inverted pendulum problem]] is posted on the course web-site as an example in the introductory section for model predictive control.
to:
where ''u'' is the force applied to the cart, {`\epsilon`} is ''m'_2_'/(m'_1_'+m'_2_')'', y is the position of the cart, ''v'' is the velocity of the cart, {`\theta`} is the angle of the pendulum relative to the cart, ''m'_1_'''=10, ''m'_2_'''=1, and ''q'' is the rate of angle change. Tune the controller to minimize the use of force applied to the cart either in the forward or reverse direction (i.e. minimize fuel consumed to perform the maneuver). Explain the tuning and the optimal solution with appropriate plots that demonstrate that the solution is optimal. A similar but [[Main/ControlTypes|non-inverted pendulum problem]] is posted on the course web-site as an example in the introductory section for model predictive control.
March 22, 2018, at 12:11 AM by 10.37.35.33 -
Added lines 1-29:
(:title Inverted Pendulum Optimal Control:)
(:keywords Python, MATLAB, linear control, model predictive control, stability, dynamic programming:)
(:description Design a model predictive controller for an inverted pendulum system with an adjustable cart. Demonstrate that the cart can perform a sequence of moves to maneuver from position y=-1.0 to y=0.0 and verify that the inverted pendulum is stationary before and after the maneuver.:)

Design a model predictive controller for an inverted pendulum system with an adjustable cart. Demonstrate that the cart can perform a sequence of moves to maneuver from position ''y''=-1.0 to ''y''=0.0 within 6.2 seconds. Verify that ''v'', {`\theta`}, and ''q'' are zero before and after the maneuver.

The inverted pendulum is described by the following dynamic equations:

\begin{bmatrix}
\dot y \\
\dot v \\
\dot \theta \\
\dot q
\end{bmatrix}

where ''u'' is the force applied to the cart, {`\epsilon`} is ''m2/(m1+m2)'', y is the position of the cart, ''v'' is the velocity of the cart, {`\theta`} is the angle of the pendulum relative to the cart, ''m1''=10, ''m2''=1, and ''q'' is the rate of angle change. Tune the controller to minimize the use of force applied to the cart either in the forward or reverse direction (i.e. minimize fuel consumed to perform the maneuver). Explain the tuning and the optimal solution with appropriate plots that demonstrate that the solution is optimal. A similar but [[Main/ControlTypes|non-inverted pendulum problem]] is posted on the course web-site as an example in the introductory section for model predictive control.

!!!! Solution

Attach:download.png [[Attach:final_solution2_files.zip|Inverted Pendulum: Solution Files]]

(:html:)
<iframe width="560" height="315" src="https://www.youtube.com/embed/iW52CoG33pk" frameborder="0" allowfullscreen></iframe>
(:htmlend:)

!!!! Reference

* Bryson, A.E., Dynamic Optimization, Addison-Wesley, 1999.