## Logical Conditions in Optimization

## Main.LogicalConditions History

Hide minor edits - Show changes to output

Changed line 61 from:

m.~~Obj~~((y+3)**2)

to:

m.Minimize((y+3)**2)

Changed line 86 from:

m.~~Obj~~((y+3)**2)

to:

m.Minimize((y+3)**2)

Deleted lines 318-336:

----

(:html:)

<div id="disqus_thread"></div>

<script type="text/javascript">

/* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */

var disqus_shortname = 'apmonitor'; // required: replace example with your forum shortname

/* * * DON'T EDIT BELOW THIS LINE * * */

(function() {

var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;

dsq.src = 'https://' + disqus_shortname + '.disqus.com/embed.js';

(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);

})();

</script>

<noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript>

<a href="https://disqus.com" class="dsq-brlink">comments powered by <span class="logo-disqus">Disqus</span></a>

(:htmlend:)

Added lines 227-251:

plt.ylabel('y')

plt.show()

(:sourceend:)

(:divend:)

(:toggle hide gekko6 button show='Show Python GEKKO if3 example':)

(:div id=gekko6:)

%width=550px%Attach:if3_gekko.png

(:source lang=python:)

import numpy as np

import matplotlib.pyplot as plt

from gekko import GEKKO

m = GEKKO(remote=False)

p = m.Param(value=np.linspace(0,10,41))

# conditional statement

y = m.if3(p-4,p**2,-0.2*(p-4)+7)

m.options.IMODE = 2

m.solve()

lbl = r'$y=\mathrm{if3}(p-4,p^2,-0.2(p-4)+7)$'

plt.plot(p,y,'bo',label=lbl)

plt.text(1,5,r'$p^2$')

plt.text(5,10,r'$-0.2 (p-4)+7$')

plt.legend(loc=4)

plt.show()

(:sourceend:)

(:divend:)

(:toggle hide gekko6 button show='Show Python GEKKO if3 example':)

(:div id=gekko6:)

%width=550px%Attach:if3_gekko.png

(:source lang=python:)

import numpy as np

import matplotlib.pyplot as plt

from gekko import GEKKO

m = GEKKO(remote=False)

p = m.Param(value=np.linspace(0,10,41))

# conditional statement

y = m.if3(p-4,p**2,-0.2*(p-4)+7)

m.options.IMODE = 2

m.solve()

lbl = r'$y=\mathrm{if3}(p-4,p^2,-0.2(p-4)+7)$'

plt.plot(p,y,'bo',label=lbl)

plt.text(1,5,r'$p^2$')

plt.text(5,10,r'$-0.2 (p-4)+7$')

plt.legend(loc=4)

Added lines 202-230:

(:toggle hide gekko5 button show='Show Python GEKKO min2 and max3 examples':)

(:div id=gekko5:)

%width=550px%Attach:min2_max3_gekko.png

(:source lang=python:)

import numpy as np

import matplotlib.pyplot as plt

from gekko import GEKKO

m = GEKKO(remote=False)

p = m.Param(value=np.linspace(10,20,21))

x = m.Var()

m.Equation(x==p)

# with MPCCs

y2 = m.min2(p,15)

# with integer variables

y3 = m.max3(p,16)

m.options.IMODE = 2

m.solve()

plt.plot(p,x,'b-',label='x')

plt.plot(p,y2,'g:',label='MPCC')

plt.plot(p,y3,'r--',label='Integer Switch')

plt.legend()

plt.xlabel('x')

plt.ylabel('y')

plt.show()

(:sourceend:)

(:divend:)

Changed lines 196-197 from:

* Max Function

~~* Min Function~~

to:

* Max Function (see Python GEKKO max2 and max3 functions)

* Min Function (see Python GEKKO min2 and min3 functions)

* Min Function (see Python GEKKO min2 and min3 functions)

Changed line 199 from:

* Piece-wise Linear ~~Functions~~

to:

* Piece-wise Linear Functions (see Python GEKKO pwl function)

Changed line 108 from:

# ~~calculate x to minimize objective~~

to:

# parameter

Changed lines 127-135 from:

* [[https://apmonitor.com/online/view_pass.php?f=abs4.apm|Example 4: Absolute Value at the Discontinuity]]

(:toggle hide gekko4 button show='Show Example 4 with Python GEKKO Binary Variable':)

(:div id=gekko4:)

'''[[https://apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization|Python GEKKO]] binary variable in optimization (Success)'''

to:

Added line 129:

# with abs3 object

Changed lines 133-143 from:

# ~~calculate ~~x ~~to minimize objective~~

x =m.~~Var~~(~~1~~.~~0~~)

#~~define new binary variable~~

intb = m.~~Var~~(~~0,lb=0,ub=1,integer=True)~~

# define y

y = m.Var()

# define equations

m.Equation((1-intb)*x <= 0)

m.Equation(intb * (-x) <= 0)

# output

m.Equation(y==(1-intb)*(-x) + intb*x)

x =

#

intb

# define y

y = m.Var()

# define equations

m.Equation((1-intb)*x <= 0)

m.Equation(intb * (-x) <= 0)

# output

m.Equation(y==(1-intb)*(-x) + intb*

to:

# parameter

x = m.Param(-0.5)

# calculate y=abs(x) with abs3

y = m.abs3(x)

x = m.Param(-0.5)

# calculate y=abs(x) with abs3

y = m.abs3(x)

Deleted line 137:

Added lines 141-171:

print('y: ' + str(y.value))

(:sourceend:)

(:divend:)

* [[https://apmonitor.com/online/view_pass.php?f=abs4.apm|Example 4: Absolute Value at the Discontinuity]]

(:toggle hide gekko4 button show='Show Example 4 with Python GEKKO Binary Variable':)

(:div id=gekko4:)

'''[[https://apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization|Python GEKKO]] binary variable in optimization (Success)'''

(:source lang=python:)

from gekko import GEKKO

# define new GEKKO model

m = GEKKO()

# calculate x to minimize objective

x = m.Var(1.0)

# define new binary variable

intb = m.Var(0,lb=0,ub=1,integer=True)

# define y

y = m.Var()

# define equations

m.Equation((1-intb)*x <= 0)

m.Equation(intb * (-x) <= 0)

# output

m.Equation(y==(1-intb)*(-x) + intb*x)

# solve with APOPT (MINLP solver)

m.options.SOLVER=1

m.solve()

# print solution

print('x: ' + str(x.value))

(:sourceend:)

(:divend:)

* [[https://apmonitor.com/online/view_pass.php?f=abs4.apm|Example 4: Absolute Value at the Discontinuity]]

(:toggle hide gekko4 button show='Show Example 4 with Python GEKKO Binary Variable':)

(:div id=gekko4:)

'''[[https://apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization|Python GEKKO]] binary variable in optimization (Success)'''

(:source lang=python:)

from gekko import GEKKO

# define new GEKKO model

m = GEKKO()

# calculate x to minimize objective

x = m.Var(1.0)

# define new binary variable

intb = m.Var(0,lb=0,ub=1,integer=True)

# define y

y = m.Var()

# define equations

m.Equation((1-intb)*x <= 0)

m.Equation(intb * (-x) <= 0)

# output

m.Equation(y==(1-intb)*(-x) + intb*x)

# solve with APOPT (MINLP solver)

m.options.SOLVER=1

m.solve()

# print solution

print('x: ' + str(x.value))

Added lines 173-188:

print('y: ' + str(y.value))

(:sourceend:)

(:source lang=python:)

# with abs3 object

from gekko import GEKKO

# define new GEKKO model

m = GEKKO()

# variable

x = m.Var(-0.5)

# calculate y=abs(x) with abs3

y = m.abs3(x)

# solve with APOPT (MINLP solver)

m.solve()

# print solution

print('x: ' + str(x.value))

(:sourceend:)

(:source lang=python:)

# with abs3 object

from gekko import GEKKO

# define new GEKKO model

m = GEKKO()

# variable

x = m.Var(-0.5)

# calculate y=abs(x) with abs3

y = m.abs3(x)

# solve with APOPT (MINLP solver)

m.solve()

# print solution

print('x: ' + str(x.value))

Deleted lines 8-9:

Added lines 231-232:

Attach:download.jpg [[Attach:mpcc_sine.zip|Sine Wave Clipping]]

Changed line 136 from:

'''[[https://apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization|Python GEKKO]] binary variable in ~~simulation~~ (Success)'''

to:

'''[[https://apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization|Python GEKKO]] binary variable in optimization (Success)'''

Changed lines 49-51 from:

(:toggle hide ~~gekko2~~ button show='Show Example 2 with Python GEKKO ABS (Failure)':)

(:div id=~~gekko2~~:)

(:div id=

to:

(:toggle hide gekko2a button show='Show Example 2 with Python GEKKO ABS (Failure)':)

(:div id=gekko2a:)

(:div id=gekko2a:)

Changed lines 74-76 from:

(:toggle hide ~~gekko2~~ button show='Show Example 2 with Python GEKKO ABS2 (Success)':)

(:div id=~~gekko2~~:)

(:div id=

to:

(:toggle hide gekko2b button show='Show Example 2 with Python GEKKO ABS2 (Success)':)

(:div id=gekko2b:)

(:div id=gekko2b:)

Changed line 143 from:

x = m.~~Param~~(~~-0~~.~~5~~)

to:

x = m.Var(1.0)

Changed lines 23-24 from:

to:

(:toggle hide gekko1 button show='Show Example 1 with Python GEKKO':)

(:div id=gekko1:)

(:div id=gekko1:)

Changed lines 30-32 from:

# use ~~abs2~~ to define a new variable

y = m.~~abs2~~(x)

# use~~abs2~~ in an equation

y = m.

# use

to:

# use abs to define a new variable

y = m.abs(x)

# use abs in an equation

y = m.abs(x)

# use abs in an equation

Changed lines 34-35 from:

m.Equation(z==m.~~abs2~~(x)+1)

to:

m.Equation(z==m.abs(x)+1)

Changed lines 43-44 from:

to:

(:divend:)

Added lines 49-51:

(:toggle hide gekko2 button show='Show Example 2 with Python GEKKO ABS (Failure)':)

(:div id=gekko2:)

(:div id=gekko2:)

Changed lines 70-71 from:

to:

(:divend:)

Added lines 74-76:

(:toggle hide gekko2 button show='Show Example 2 with Python GEKKO ABS2 (Success)':)

(:div id=gekko2:)

(:div id=gekko2:)

Changed lines 95-96 from:

to:

(:divend:)

Added lines 100-130:

(:toggle hide gekko3 button show='Show Example 3 with Python GEKKO Binary Variable':)

(:div id=gekko3:)

'''[[https://apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization|Python GEKKO]] binary variable in simulation (Success)'''

(:source lang=python:)

from gekko import GEKKO

# define new GEKKO model

m = GEKKO()

# calculate x to minimize objective

x = m.Param(-0.5)

# define new binary variable

intb = m.Var(0,lb=0,ub=1,integer=True)

# define y

y = m.Var()

# define equations

m.Equation((1-intb)*x <= 0)

m.Equation(intb * (-x) <= 0)

# output

m.Equation(y==(1-intb)*(-x) + intb*x)

# solve with APOPT (MINLP solver)

m.options.SOLVER=1

m.solve()

# print solution

print('x: ' + str(x.value))

print('intb: ' + str(intb.value))

print('y: ' + str(y.value))

(:sourceend:)

(:divend:)

Added lines 132-161:

(:toggle hide gekko4 button show='Show Example 4 with Python GEKKO Binary Variable':)

(:div id=gekko4:)

'''[[https://apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization|Python GEKKO]] binary variable in simulation (Success)'''

(:source lang=python:)

from gekko import GEKKO

# define new GEKKO model

m = GEKKO()

# calculate x to minimize objective

x = m.Param(-0.5)

# define new binary variable

intb = m.Var(0,lb=0,ub=1,integer=True)

# define y

y = m.Var()

# define equations

m.Equation((1-intb)*x <= 0)

m.Equation(intb * (-x) <= 0)

# output

m.Equation(y==(1-intb)*(-x) + intb*x)

# solve with APOPT (MINLP solver)

m.options.SOLVER=1

m.solve()

# print solution

print('x: ' + str(x.value))

print('intb: ' + str(intb.value))

print('y: ' + str(y.value))

(:sourceend:)

(:divend:)

Changed lines 23-24 from:

'''GEKKO ABS function (abs2)'''

to:

'''[[https://apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization|Python GEKKO]] ABS function (abs2)'''

Changed lines 48-88 from:

to:

'''[[https://apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization|Python GEKKO]] ABS function in optimization (Failure)'''

(:source lang=python:)

from gekko import GEKKO

# define new GEKKO model

m = GEKKO()

# calculate x to minimize objective

x = m.Var(1)

# use abs to define y

y = m.abs(x)

# define objective to minimize

m.Obj((y+3)**2)

# solve

m.solve()

# print solution

print('x: ' + str(x.value))

print('y: ' + str(y.value))

(:sourceend:)

In this case, the IPOPT solver reaches the default maximum number of iterations (100) and reports a failure to converge. However, if another form with continuous first and second derivatives ('''abs2''' in GEKKO) is used, a solution is found.

'''[[https://apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization|Python GEKKO]] ABS2 function in optimization (Success)'''

(:source lang=python:)

from gekko import GEKKO

# define new GEKKO model

m = GEKKO()

# calculate x to minimize objective

x = m.Var(1)

# use abs2 to define y

y = m.abs2(x)

# define objective to minimize

m.Obj((y+3)**2)

# solve

m.solve()

# print solution

print('x: ' + str(x.value))

print('y: ' + str(y.value))

(:sourceend:)

Also, if the problem is reformulated with an additional binary variable ''intb'', the optimizer can find a solution to the same problem that failed with the abs(x) operator. The value of ''intb'' is a binary variable that can be either ''0'' or ''1''. The value of ''intb'' is zero when ''x<0'' and is one when ''x>=0''. The following two examples show that this reformulation allows the solver to quickly find a solution either away from or at the discontinuity.

(:source lang=python:)

from gekko import GEKKO

# define new GEKKO model

m = GEKKO()

# calculate x to minimize objective

x = m.Var(1)

# use abs to define y

y = m.abs(x)

# define objective to minimize

m.Obj((y+3)**2)

# solve

m.solve()

# print solution

print('x: ' + str(x.value))

print('y: ' + str(y.value))

(:sourceend:)

In this case, the IPOPT solver reaches the default maximum number of iterations (100) and reports a failure to converge. However, if another form with continuous first and second derivatives ('''abs2''' in GEKKO) is used, a solution is found.

'''[[https://apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization|Python GEKKO]] ABS2 function in optimization (Success)'''

(:source lang=python:)

from gekko import GEKKO

# define new GEKKO model

m = GEKKO()

# calculate x to minimize objective

x = m.Var(1)

# use abs2 to define y

y = m.abs2(x)

# define objective to minimize

m.Obj((y+3)**2)

# solve

m.solve()

# print solution

print('x: ' + str(x.value))

print('y: ' + str(y.value))

(:sourceend:)

Also, if the problem is reformulated with an additional binary variable ''intb'', the optimizer can find a solution to the same problem that failed with the abs(x) operator. The value of ''intb'' is a binary variable that can be either ''0'' or ''1''. The value of ''intb'' is zero when ''x<0'' and is one when ''x>=0''. The following two examples show that this reformulation allows the solver to quickly find a solution either away from or at the discontinuity.

Added lines 22-42:

'''GEKKO ABS function (abs2)'''

(:source lang=python:)

from gekko import GEKKO

m = GEKKO()

x = m.Param(-1)

# use abs2 to define a new variable

y = m.abs2(x)

# use abs2 in an equation

z = m.Var()

m.Equation(z==m.abs2(x)+1)

# solve

m.solve()

print('x: ' + str(x.value))

print('y: ' + str(y.value))

print('z: ' + str(z.value))

(:sourceend:)

Changed line 9 from:

Attach:download.~~png~~ [[Attach:mpcc_sine.zip|Sine Wave Clipping]]

to:

Attach:download.jpg [[Attach:mpcc_sine.zip|Sine Wave Clipping]]

Changed lines 7-15 from:

Conditional statements can cause problems for gradient-based optimization algorithms because they are often posed in a form that gives discontinuous functions, first derivatives, or second derivatives. For example, the absolute value operator is a continuous function but gives a discontinuous first derivative at the origin.

to:

Conditional statements can cause problems for gradient-based optimization algorithms because they are often posed in a form that gives discontinuous functions, first derivatives, or second derivatives.

Attach:download.png [[Attach:mpcc_sine.zip|Sine Wave Clipping]]

(:html:)

<iframe width="560" height="315" src="https://www.youtube.com/embed/NPUNixLGlh0" frameborder="0" allowfullscreen></iframe>

(:htmlend:)

For example, the absolute value operator is a continuous function but gives a discontinuous first derivative at the origin.

Attach:download.png [[Attach:mpcc_sine.zip|Sine Wave Clipping]]

(:html:)

<iframe width="560" height="315" src="https://www.youtube.com/embed/NPUNixLGlh0" frameborder="0" allowfullscreen></iframe>

(:htmlend:)

For example, the absolute value operator is a continuous function but gives a discontinuous first derivative at the origin.

Changed lines 83-93 from:

!!!! ~~MPECs~~: Mathematical Programs with ~~Equilibrium~~ Constraints

Mathematical Programs with~~Equilibrium~~ Constraints (~~MPECs~~) are formulations that can also be used to model certain classes of discrete events. ~~MPECs~~ can be more efficient than solving mixed integer formulations of the optimization problems because it avoids the combinatorial difficulties of searching for optimal discrete variables.

* [[https://apmonitor.com/online/view_pass.php?f=abs_mpec.apm|ABS: Absolute Value Operator (~~MPEC~~ Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=max_mpec.apm|MAX: Maximum Operator (~~MPEC~~ Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=min_mpec.apm|MIN: Minimum Operator (~~MPEC~~ Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=pwl0.apm|Piecewise Linear Function without Object Use (~~MPEC~~ Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=pwl1.apm|Piecewise Linear Function with PWL Object (~~MPEC~~ Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=pwl2.apm|Piecewise Linear Function with LOOKUP and PWL Objects (~~MPEC~~ Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=sign_mpec.apm|SIGN: Signum Operator (~~MPEC~~ Form)]]

Mathematical Programs with

* [[https://apmonitor.com/online/view_pass.php?f=abs_mpec.apm|ABS: Absolute Value Operator (

* [[https://apmonitor.com/online/view_pass.php?f=max_mpec.apm|MAX: Maximum Operator (

* [[https://apmonitor.com/online/view_pass.php?f=min_mpec.apm|MIN: Minimum Operator (

* [[https://apmonitor.com/online/view_pass.php?f=pwl0.apm|Piecewise Linear Function without Object Use (

* [[https://apmonitor.com/online/view_pass.php?f=pwl1.apm|Piecewise Linear Function with PWL Object (

* [[https://apmonitor.com/online/view_pass.php?f=pwl2.apm|Piecewise Linear Function with LOOKUP and PWL Objects (

* [[https://apmonitor.com/online/view_pass.php?f=sign_mpec.apm|SIGN: Signum Operator (

to:

!!!! MPCCs: Mathematical Programs with Complementarity Constraints

Mathematical Programs with Complementarity Constraints (MPCCs) are formulations that can also be used to model certain classes of discrete events. MPCCs can be more efficient than solving mixed integer formulations of the optimization problems because it avoids the combinatorial difficulties of searching for optimal discrete variables.

* [[https://apmonitor.com/online/view_pass.php?f=abs_mpec.apm|ABS: Absolute Value Operator (MPCC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=max_mpec.apm|MAX: Maximum Operator (MPCC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=min_mpec.apm|MIN: Minimum Operator (MPCC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=pwl0.apm|Piecewise Linear Function without Object Use (MPCC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=pwl1.apm|Piecewise Linear Function with PWL Object (MPCC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=pwl2.apm|Piecewise Linear Function with LOOKUP and PWL Objects (MPCC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=sign_mpec.apm|SIGN: Signum Operator (MPCC Form)]]

Mathematical Programs with Complementarity Constraints (MPCCs) are formulations that can also be used to model certain classes of discrete events. MPCCs can be more efficient than solving mixed integer formulations of the optimization problems because it avoids the combinatorial difficulties of searching for optimal discrete variables.

* [[https://apmonitor.com/online/view_pass.php?f=abs_mpec.apm|ABS: Absolute Value Operator (MPCC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=max_mpec.apm|MAX: Maximum Operator (MPCC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=min_mpec.apm|MIN: Minimum Operator (MPCC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=pwl0.apm|Piecewise Linear Function without Object Use (MPCC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=pwl1.apm|Piecewise Linear Function with PWL Object (MPCC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=pwl2.apm|Piecewise Linear Function with LOOKUP and PWL Objects (MPCC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=sign_mpec.apm|SIGN: Signum Operator (MPCC Form)]]

Changed line 37 from:

* Mathematical Programs with Equilibrium Constraints (MPECs)

to:

* Mathematical Programs with Equilibrium/Complementarity Constraints (MPECs/MPCCs)

Added line 29:

* [[https://apmonitor.com/me575/index.php/Main/MiniMax|Minimax and Maximin]]

Changed lines 33-34 from:

Two popular methods for reformulation of the conditional statements~~ or operators~~ is to either use:

to:

Two popular methods for reformulation of the conditional statements is to either use:

Changed lines 38-39 from:

Each of these techniques are described below with a number of examples~~. The approach taken above with the ABS function used a single binary variable. The ABS function can also be expressed as an MPEC as shown below~~.

to:

Each of these techniques are described below with a number of examples.

Deleted lines 41-42:

Added lines 44-49:

The approach taken above with the ABS function used a single binary variable.

* [[https://apmonitor.com/online/view_pass.php?f=abs4.apm|ABS: Absolute Value Operator (Integer Form)]]

----

* [[https://apmonitor.com/online/view_pass.php?f=abs4.apm|ABS: Absolute Value Operator (Integer Form)]]

----

Added lines 68-81:

Piece-wise Linear (PWL) functions are one method to approximate any nonlinear function. The following example programs demonstrate a piece-wise linear function with binary decision variables. The sum of the binary decision variables is required to be one, meaning that only one of the linear approximations can be active at a time.

* [[https://apmonitor.com/online/view_pass.php?f=pwl3.apm|Simulate Piecewise Linear Function with Binary Decision Variables]]

The value of ''x'' is now declared as a decision variable and is adjusted by the optimizer to minimize the objective function that is equal to ''y''. By inspection, the optimal value is at ''x=2'' giving a result of ''y=0''.

* [[https://apmonitor.com/online/view_pass.php?f=pwl4.apm|Optimize Piecewise Linear Function with Binary Decision Variables]]

Attach:pwl_function.png

The example shown above demonstrates a PWL function with one input and one output but it can also be extended to cases with multiple inputs and multiple outputs.

----

* [[https://apmonitor.com/online/view_pass.php?f=pwl3.apm|Simulate Piecewise Linear Function with Binary Decision Variables]]

The value of ''x'' is now declared as a decision variable and is adjusted by the optimizer to minimize the objective function that is equal to ''y''. By inspection, the optimal value is at ''x=2'' giving a result of ''y=0''.

* [[https://apmonitor.com/online/view_pass.php?f=pwl4.apm|Optimize Piecewise Linear Function with Binary Decision Variables]]

Attach:pwl_function.png

The example shown above demonstrates a PWL function with one input and one output but it can also be extended to cases with multiple inputs and multiple outputs.

----

Added lines 89-91:

* [[https://apmonitor.com/online/view_pass.php?f=pwl0.apm|Piecewise Linear Function without Object Use (MPEC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=pwl1.apm|Piecewise Linear Function with PWL Object (MPEC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=pwl2.apm|Piecewise Linear Function with LOOKUP and PWL Objects (MPEC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=pwl1.apm|Piecewise Linear Function with PWL Object (MPEC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=pwl2.apm|Piecewise Linear Function with LOOKUP and PWL Objects (MPEC Form)]]

Changed lines 44-45 from:

to:

----

The following examples demonstrate the use of conditional statements to switch between different functions. In this case, the function ''y=6*exp(-x)+1'' if the value of ''x<3'' else the function ''y=x'' is chosen when ''x>=3''. The simulate example verifies that the correct value is reported with a fixed value of ''x''. The second example shows how ''x'' can be adjusted to find a minimum value of ''y'' or ''f(x)''.

The following examples demonstrate the use of conditional statements to switch between different functions. In this case, the function ''y=6*exp(-x)+1'' if the value of ''x<3'' else the function ''y=x'' is chosen when ''x>=3''. The simulate example verifies that the correct value is reported with a fixed value of ''x''. The second example shows how ''x'' can be adjusted to find a minimum value of ''y'' or ''f(x)''.

Changed lines 51-52 from:

Attach:if_~~elseif_~~else_~~3~~_functions.png

to:

Attach:if_else_2_functions.png

----

With three functions an if...elseif...else structure may be used. The following two examples show how to implement these conditional statements to achieve continuous first and second derivatives.

----

With three functions an if...elseif...else structure may be used. The following two examples show how to implement these conditional statements to achieve continuous first and second derivatives.

Added lines 60-62:

Attach:if_elseif_else_3_functions.png

----

----

Changed lines 15-16 from:

The solver can still converge because the solution is away from the discontinuity. In Example 2, the value of ''x'' is now determined by the optimizer. The objective function to minimize (y+3)'^2^' means that the optimal solution is at ''x=0'' and ''y~~=~~=0''. However, when the solution is at the discontinuity, a gradient based solver will often fail to converge.

to:

The solver can still converge because the solution is away from the discontinuity. In Example 2, the value of ''x'' is now determined by the optimizer. The objective function to minimize (y+3)'^2^' means that the optimal solution is at ''x=0'' and ''y=0''. However, when the solution is at the discontinuity, a gradient based solver will often fail to converge.

Changed lines 40-42 from:

!!!! Logical Conditions with ~~Integer~~ Variables

to:

!!!! Logical Conditions with Binary Variables

* [[https://apmonitor.com/online/view_pass.php?f=abs4.apm|ABS: Absolute Value Operator (Integer Form)]]

Attach:if_else_2_functions.png

* [[https://apmonitor.com/online/view_pass.php?f=if1a.apm|IF: Conditional Statement with Two Functions - Simulate (Integer Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=if1b.apm|IF: Conditional Statement with Two Functions - Optimize (Integer Form)]]

Attach:if_elseif_else_3_functions.png

* [[https://apmonitor.com/online/view_pass.php?f=if2a.apm|IF: Conditional Statement with Three Functions - Simulate (Integer Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=if2b.apm|IF: Conditional Statement with Three Functions - Optimize (Integer Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=abs4.apm|ABS: Absolute Value Operator (Integer Form)]]

Attach:if_else_2_functions.png

* [[https://apmonitor.com/online/view_pass.php?f=if1a.apm|IF: Conditional Statement with Two Functions - Simulate (Integer Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=if1b.apm|IF: Conditional Statement with Two Functions - Optimize (Integer Form)]]

Attach:if_elseif_else_3_functions.png

* [[https://apmonitor.com/online/view_pass.php?f=if2a.apm|IF: Conditional Statement with Three Functions - Simulate (Integer Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=if2b.apm|IF: Conditional Statement with Three Functions - Optimize (Integer Form)]]

Changed line 30 from:

* [[https://en.wikipedia.org/wiki/Signum_function|~~Signum Function~~]]

to:

* Signum Function ([[https://en.wikipedia.org/wiki/Signum_function|What is Signum?]])

Changed lines 5-6 from:

to:

!!! Conditional Statements in Optimization

Changed lines 13-14 from:

* [[https://apmonitor.com/online/view_pass.php?f=abs1.apm|Example 1: Absolute Value Operator]]

to:

* [[https://apmonitor.com/online/view_pass.php?f=abs1.apm|Example 1: Absolute Value Operator Away from the Discontinuity]]

Changed lines 17-22 from:

* [[https://apmonitor.com/online/view_pass.php?f=~~abs1~~.apm|Example 2: Absolute Value Operator ~~in Optimization~~]]

to:

* [[https://apmonitor.com/online/view_pass.php?f=abs2.apm|Example 2: Absolute Value Operator at the Discontinuity]]

In this case, the IPOPT solver reaches the default maximum number of iterations (100) and reports a failure to converge. Now if the problem is reformulated with an additional variable ''intb'', the optimizer can find a solution to the same problem that failed with the abs(x) operator. The value of ''intb'' is a binary variable that can be either ''0'' or ''1''. The value of ''intb'' is zero when ''x<0'' and is one when ''x>=0''. The following two examples show that this reformulation allows the solver to quickly find a solution either away from or at the discontinuity.

* [[https://apmonitor.com/online/view_pass.php?f=abs3.apm|Example 3: Absolute Value away from the Discontinuity]]

* [[https://apmonitor.com/online/view_pass.php?f=abs4.apm|Example 4: Absolute Value at the Discontinuity]]

In this case, the IPOPT solver reaches the default maximum number of iterations (100) and reports a failure to converge. Now if the problem is reformulated with an additional variable ''intb'', the optimizer can find a solution to the same problem that failed with the abs(x) operator. The value of ''intb'' is a binary variable that can be either ''0'' or ''1''. The value of ''intb'' is zero when ''x<0'' and is one when ''x>=0''. The following two examples show that this reformulation allows the solver to quickly find a solution either away from or at the discontinuity.

* [[https://apmonitor.com/online/view_pass.php?f=abs3.apm|Example 3: Absolute Value away from the Discontinuity]]

* [[https://apmonitor.com/online/view_pass.php?f=abs4.apm|Example 4: Absolute Value at the Discontinuity]]

Added line 25:

Changed lines 29-62 from:

* ~~Sign Operator~~

* Most other discontinuous functions

!!!! MPEC: Mathematical Programs with Equilibrium Constraints

Mathematical Programs with Equilibrium Constraints (MPECs) are formulations that can be used to model certain classes of discrete events.MPECs ~~can be more efficient than solving mixed integer formulations~~ of ~~the optimization problems because it avoids~~ the ~~combinatorial difficulties of searching for optimal discrete variables~~.

----

!!! SIGN Operator (MPEC Form)

* %list list-page% [[Attach:sign.apm | SIGN Operator Example]]

----

!!! Absolute Value (ABS) Operator

* %list list-page% [[Attach:~~abs.apm | ABS Operator Example]]~~

----

!!! Minimum Selector (MIN) Operator

* %list list-page% [[Attach:min.apm | MIN Operator Example]]

----

!!! Maximum Selector (MAX) Operator

* %list list-page% [[Attach:max.apm | MAX Operator Example]]

* Most other discontinuous

!!!! MPEC: Mathematical Programs with Equilibrium Constraints

Mathematical Programs with Equilibrium Constraints (MPECs) are formulations that can be used to model certain classes of discrete events.

----

!!! SIGN Operator (MPEC Form)

* %list list-page% [[Attach:sign.apm | SIGN Operator Example]]

----

!!! Absolute Value (ABS) Operator

* %list list-page% [[Attach

----

!!! Minimum Selector (MIN) Operator

* %list list-page% [[Attach:min.apm | MIN Operator Example]]

----

!!! Maximum Selector (MAX) Operator

* %list list-page% [[Attach:max.apm | MAX Operator Example

to:

* Piece-wise Linear Functions

* [[https://en.wikipedia.org/wiki/Signum_function|Signum Function]]

* Other discontinuous functions

Two popular methods for reformulation of the conditional statements or operators is to either use:

* Binary or integer variables

* Mathematical Programs with Equilibrium Constraints (MPECs)

Each of these techniques are described below with a number of examples. The approach taken above with the ABS function used a single binary variable. The ABS function can also be expressed as an MPEC as shown below.

!!!! Logical Conditions with Integer Variables

!!!! MPECs: Mathematical Programs with Equilibrium Constraints

Mathematical Programs with Equilibrium Constraints (MPECs) are formulations that can also be used to model certain classes of discrete events. MPECs can be more efficient than solving mixed integer formulations of the optimization problems because it avoids the combinatorial difficulties of searching for optimal discrete variables.

* [[https://apmonitor.com/online/view_pass.php?f=abs_mpec.apm|ABS: Absolute Value Operator (MPEC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=max_mpec.apm|MAX: Maximum Operator (MPEC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=min_mpec.apm|MIN: Minimum Operator (MPEC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=sign_mpec.apm|SIGN: Signum Operator (MPEC Form)]]

* [[https://en.wikipedia.org/wiki/Signum_function|Signum Function]]

* Other discontinuous functions

Two popular methods for reformulation of the conditional statements or operators is to either use:

* Binary or integer variables

* Mathematical Programs with Equilibrium Constraints (MPECs)

Each of these techniques are described below with a number of examples. The approach taken above with the ABS function used a single binary variable. The ABS function can also be expressed as an MPEC as shown below.

!!!! Logical Conditions with Integer Variables

!!!! MPECs: Mathematical Programs with Equilibrium Constraints

Mathematical Programs with Equilibrium Constraints (MPECs) are formulations that can also be used to model certain classes of discrete events. MPECs can be more efficient than solving mixed integer formulations of the optimization problems because it avoids the combinatorial difficulties of searching for optimal discrete variables.

* [[https://apmonitor.com/online/view_pass.php?f=abs_mpec.apm|ABS: Absolute Value Operator (MPEC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=max_mpec.apm|MAX: Maximum Operator (MPEC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=min_mpec.apm|MIN: Minimum Operator (MPEC Form)]]

* [[https://apmonitor.com/online/view_pass.php?f=sign_mpec.apm|SIGN: Signum Operator (MPEC Form)]]

Changed lines 11-30 from:

Minimizing this function can cause a problem for solvers that rely on first and second derivative information to determine search directions ~~for~~ each iteration.

to:

Minimizing this function can cause a problem for solvers that rely on first and second derivative information to determine search directions at each iteration. As an exercise, solve the following model that uses the absolute value operator.

* [[https://apmonitor.com/online/view_pass.php?f=abs1.apm|Example 1: Absolute Value Operator]]

The solver can still converge because the solution is away from the discontinuity. In Example 2, the value of ''x'' is now determined by the optimizer. The objective function to minimize (y+3)'^2^' means that the optimal solution is at ''x=0'' and ''y==0''. However, when the solution is at the discontinuity, a gradient based solver will often fail to converge.

* [[https://apmonitor.com/online/view_pass.php?f=abs1.apm|Example 2: Absolute Value Operator in Optimization]]

The above examples demonstrate this concept with the absolute value operator. The same techniques can be applied to:

* If...Else Statements

* Max Function

* Min Function

* Sign Operator

* Most other discontinuous functions

* [[https://apmonitor.com/online/view_pass.php?f=abs1.apm|Example 1: Absolute Value Operator]]

The solver can still converge because the solution is away from the discontinuity. In Example 2, the value of ''x'' is now determined by the optimizer. The objective function to minimize (y+3)'^2^' means that the optimal solution is at ''x=0'' and ''y==0''. However, when the solution is at the discontinuity, a gradient based solver will often fail to converge.

* [[https://apmonitor.com/online/view_pass.php?f=abs1.apm|Example 2: Absolute Value Operator in Optimization]]

The above examples demonstrate this concept with the absolute value operator. The same techniques can be applied to:

* If...Else Statements

* Max Function

* Min Function

* Sign Operator

* Most other discontinuous functions

Added lines 1-60:

(:title Logical Conditions in Optimization:)

(:keywords if conditions, maximum, minimum, max operator, min operator, sign operator, APMonitor, summation:)

(:description Formulate conditional statements in numerical optimization so that the problem can be solved with gradient based approaches.:)

!!!! Conditional Statements in Optimization

Conditional statements can cause problems for gradient-based optimization algorithms because they are often posed in a form that gives discontinuous functions, first derivatives, or second derivatives. For example, the absolute value operator is a continuous function but gives a discontinuous first derivative at the origin.

Attach:absolute_value_1st_deriv.png

Minimizing this function can cause a problem for solvers that rely on first and second derivative information to determine search directions for each iteration.

!!!! MPEC: Mathematical Programs with Equilibrium Constraints

Mathematical Programs with Equilibrium Constraints (MPECs) are formulations that can be used to model certain classes of discrete events. MPECs can be more efficient than solving mixed integer formulations of the optimization problems because it avoids the combinatorial difficulties of searching for optimal discrete variables.

----

!!! SIGN Operator (MPEC Form)

* %list list-page% [[Attach:sign.apm | SIGN Operator Example]]

----

!!! Absolute Value (ABS) Operator

* %list list-page% [[Attach:abs.apm | ABS Operator Example]]

----

!!! Minimum Selector (MIN) Operator

* %list list-page% [[Attach:min.apm | MIN Operator Example]]

----

!!! Maximum Selector (MAX) Operator

* %list list-page% [[Attach:max.apm | MAX Operator Example]]

----

(:html:)

<div id="disqus_thread"></div>

<script type="text/javascript">

/* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */

var disqus_shortname = 'apmonitor'; // required: replace example with your forum shortname

/* * * DON'T EDIT BELOW THIS LINE * * */

(function() {

var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;

dsq.src = 'https://' + disqus_shortname + '.disqus.com/embed.js';

(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);

})();

</script>

<noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript>

<a href="https://disqus.com" class="dsq-brlink">comments powered by <span class="logo-disqus">Disqus</span></a>

(:htmlend:)

(:keywords if conditions, maximum, minimum, max operator, min operator, sign operator, APMonitor, summation:)

(:description Formulate conditional statements in numerical optimization so that the problem can be solved with gradient based approaches.:)

!!!! Conditional Statements in Optimization

Conditional statements can cause problems for gradient-based optimization algorithms because they are often posed in a form that gives discontinuous functions, first derivatives, or second derivatives. For example, the absolute value operator is a continuous function but gives a discontinuous first derivative at the origin.

Attach:absolute_value_1st_deriv.png

Minimizing this function can cause a problem for solvers that rely on first and second derivative information to determine search directions for each iteration.

!!!! MPEC: Mathematical Programs with Equilibrium Constraints

Mathematical Programs with Equilibrium Constraints (MPECs) are formulations that can be used to model certain classes of discrete events. MPECs can be more efficient than solving mixed integer formulations of the optimization problems because it avoids the combinatorial difficulties of searching for optimal discrete variables.

----

!!! SIGN Operator (MPEC Form)

* %list list-page% [[Attach:sign.apm | SIGN Operator Example]]

----

!!! Absolute Value (ABS) Operator

* %list list-page% [[Attach:abs.apm | ABS Operator Example]]

----

!!! Minimum Selector (MIN) Operator

* %list list-page% [[Attach:min.apm | MIN Operator Example]]

----

!!! Maximum Selector (MAX) Operator

* %list list-page% [[Attach:max.apm | MAX Operator Example]]

----

(:html:)

<div id="disqus_thread"></div>

<script type="text/javascript">

/* * * CONFIGURATION VARIABLES: EDIT BEFORE PASTING INTO YOUR WEBPAGE * * */

var disqus_shortname = 'apmonitor'; // required: replace example with your forum shortname

/* * * DON'T EDIT BELOW THIS LINE * * */

(function() {

var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;

dsq.src = 'https://' + disqus_shortname + '.disqus.com/embed.js';

(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);

})();

</script>

<noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript>

<a href="https://disqus.com" class="dsq-brlink">comments powered by <span class="logo-disqus">Disqus</span></a>

(:htmlend:)