Equality Constraints

Equality constraints are mathematical conditions that require certain functions of the design variables to equal specific values in optimization problems. They represent exact requirements that must be satisfied by any feasible solution.

Mathematical Formulation

An optimization problem with equality constraints can be expressed as:

\begin{align} \min_{\theta} \quad & L(\theta) \ \text{subject to} \quad & E_j(\theta) = 0, \quad j = 1,2,\ldots,m \end{align}

where:

• $\theta \in {\mathbb{R}}^D$ is the design vector
• $L(\theta )$ is the objective function to be minimized
• $E_j(\theta )$ are the equality constraint functions
• $m$ is the number of equality constraints

Types of Equality Constraints

Linear Equality Constraints

Constraints that are linear functions of the design variables:

$E_j(\theta )={\mathbf{a}}_j^T\theta -b_j=0$

where ${\mathbf{a}}_j$ is a constant vector and $b_j$ is a constant scalar.

Examples:

• Budget constraint: ${\sum }_{i=1}^D{\theta }_i=1$
• Material conservation: ${\sum }_{i=1}^D{w}_i{\theta }_i=W_{\text{total}}$
• Physical equilibrium: ${\sum }_{i=1}^D{F}_i{\theta }_i=0$

Nonlinear Equality Constraints

Constraints that have nonlinear relationships with the design variables:

$E_j(\theta )=g_j(\theta )=0$

where $g_j$ is a nonlinear function.

Examples:

• Unit sphere constraint: ${\sum }_{i=1}^D{\theta }_i^2-1=0$
• Energy conservation: $f({\theta }_1,{\theta }_2,\dots ,{\theta }_D)-E_0=0$
• Geometric requirements: $\text{area}(\theta )-A_0=0$

Geometric Interpretation

Equality constraints define hypersurfaces in the design space:

• Each constraint $E_j(\theta )=0$ defines a $(D-1)$-dimensional surface in the $D$-dimensional design space
• The feasible region is the intersection of all these surfaces
• If there are $m$ independent equality constraints, the feasible region has dimension $D-m$

When $m=D$, the feasible region typically consists of isolated points. When $m>D$, the system is typically overdetermined with no feasible solutions unless the constraints are dependent.

Solution Methods

Direct Substitution

For simple problems with explicit constraints:

1. Solve the constraint equations for a subset of the variables
2. Substitute these expressions into the objective function
3. Optimize the reduced problem in fewer variables

This approach reduces a constrained optimization problem with $m$ equality constraints to an unconstrained problem with $D-m$ variables.

Example: For the constraint ${\theta }_1+{\theta }_2+{\theta }_3=1$, we can substitute ${\theta }_3=1-{\theta }_1-{\theta }_2$ and optimize with respect to just ${\theta }_1$ and ${\theta }_2$.

Lagrange Multipliers

Introduce Lagrange multipliers to incorporate the constraints:

$\mathcal{L}(\theta ,\lambda )=L(\theta )-{\sum }_{j=1}^m{\lambda }_j{E}_j(\theta )$

The necessary conditions for optimality are:

• ${\nabla }_{\theta }\mathcal{L}(\theta ,\lambda )=0$
• $E_j(\theta )=0$ for all $j=1,2,\dots ,m$

This yields a system of $D+m$ equations in $D+m$ unknowns ($\theta$ and $\lambda$).

Penalty Methods

Convert the constrained problem into an unconstrained one by adding penalty terms:

${\min }_{\theta }L(\theta )+\mu {\sum }_{j=1}^m{E}_j(\theta {)}^2$

where $\mu >0$ is a penalty parameter that is gradually increased.

Augmented Lagrangian Method

Combine Lagrange multipliers with penalty terms:

${\min }_{\theta }L(\theta )-{\sum }_{j=1}^m{\lambda }_j{E}_j(\theta )+\frac{\mu }{2}{\sum }_{j=1}^m{E}_j(\theta {)}^2$

Null Space Method

For problems with linear constraints:

1. Express the constraints in matrix form: $\mathbf{A}\theta =\mathbf{b}$
2. Find a basis $\mathbf{Z}$ for the null space of $\mathbf{A}$
3. Parameterize the feasible region as $\theta ={\theta }_0+\mathbf{Z}v$, where $\mathbf{A}{\theta }_0=\mathbf{b}$
4. Optimize with respect to the reduced variables $v$

Constraint Qualification

For the method of Lagrange multipliers to provide necessary conditions for optimality, the equality constraints must satisfy certain regularity conditions known as constraint qualifications:

The most common constraint qualification is the Linear Independence Constraint Qualification (LICQ), which requires that the gradients of the active constraints are linearly independent at the solution:

$\text{rank}(\nabla E_1({\theta }^{\ast }),\nabla E_2({\theta }^{\ast }),\dots ,\nabla E_m({\theta }^{\ast }))=m$

If this condition fails, the Lagrange multipliers may not exist or may not be unique.

Challenges with Equality Constraints

Feasibility

Finding an initial feasible point (satisfying all constraints) can be challenging, especially for nonlinear constraints. Common approaches include:

• Starting from a known feasible point
• Solving a phase-1 problem to find a feasible point
• Using constraint relaxation techniques

Over-Constraining

If too many equality constraints are imposed ($m>D$ independent constraints), the problem becomes overconstrained and typically has no solution. In engineering applications, this may require:

• Relaxing some constraints
• Converting some equality constraints to inequality constraints
• Using a least-squares approach to find a solution that minimizes constraint violation

Numerical Stability

Enforcing equality constraints exactly can lead to numerical difficulties:

• Accumulation of numerical errors
• Ill-conditioning in the Lagrangian system
• Difficulties with constraint linearization

Engineering Applications

Structural Design

• Equilibrium equations: $\sum \text{Forces}=0$, $\sum \text{Moments}=0$
• Displacement compatibility: $u_i=u_j$ at connected nodes
• Material conservation: $\sum \text{Volume}=\text{Total Volume}$

Chemical Engineering

• Mass balance: $\sum \text{Input flows}=\sum \text{Output flows}$
• Energy balance: $\sum \text{Energy in}=\sum \text{Energy out}$
• Chemical equilibrium: $K=\frac{[C][D]}{[A][B]}$ for reaction $A+B\rightleftharpoons C+D$

Electrical Engineering

• Kirchhoff’s laws: $\sum \text{Currents at a node}=0$, $\sum \text{Voltage drops in a loop}=0$
• Power balance: $P_{\text{generated}}=P_{\text{consumed}}+P_{\text{lost}}$

Mechanical Design

• Kinematic constraints: $\text{Distance}(A,B)=L$
• Gear ratios: ${\omega }_1{r}_1={\omega }_2{r}_2$
• Conservation of momentum: $m_1{v}_1+m_2{v}_2=\text{constant}$

Example: Optimization with Equality Constraints

Consider the problem:

\begin{align} \min_{\theta_1, \theta_2, \theta_3} \quad & \theta_1^2 + \theta_2^2 + \theta_3^2 \ \text{subject to} \quad & \theta_1 + \theta_2 + \theta_3 = 1 \end{align}

This is seeking the point on the plane ${\theta }_1+{\theta }_2+{\theta }_3=1$ that is closest to the origin.

Solution using Lagrange Multipliers

1. Form the Lagrangian: $\mathcal{L}({\theta }_1,{\theta }_2,{\theta }_3,\lambda )={\theta }_1^2+{\theta }_2^2+{\theta }_3^2-\lambda ({\theta }_1+{\theta }_2+{\theta }_3-1)$

2. Take derivatives and set to zero: $\frac{\partial \mathcal{L}}{\partial {\theta }_1}=2{\theta }_1-\lambda =0⟹{\theta }_1=\frac{\lambda }{2}$ $\frac{\partial \mathcal{L}}{\partial {\theta }_2}=2{\theta }_2-\lambda =0⟹{\theta }_2=\frac{\lambda }{2}$ $\frac{\partial \mathcal{L}}{\partial {\theta }_3}=2{\theta }_3-\lambda =0⟹{\theta }_3=\frac{\lambda }{2}$ $\frac{\partial \mathcal{L}}{\partial \lambda }=-({\theta }_1+{\theta }_2+{\theta }_3-1)=0⟹{\theta }_1+{\theta }_2+{\theta }_3=1$

3. From the first three equations, we have ${\theta }_1={\theta }_2={\theta }_3=\frac{\lambda }{2}$

4. Substitute into the constraint: $\frac{\lambda }{2}+\frac{\lambda }{2}+\frac{\lambda }{2}=1⟹\frac{3\lambda }{2}=1⟹\lambda =\frac{2}{3}$

5. Therefore: ${\theta }_1={\theta }_2={\theta }_3=\frac{1}{3}$

6. The solution is ${\theta }^{\ast }=(\frac{1}{3},\frac{1}{3},\frac{1}{3})$, which is the point on the plane ${\theta }_1+{\theta }_2+{\theta }_3=1$ closest to the origin.

Related Pages

• Lagrange Multipliers
• KKT Conditions
• Inequality Constraints
• Penalty Methods
• Augmented Lagrangian Method
• Constrained Optimization