Functional Optimisation

Introduction

Functional optimization deals with finding functions (rather than finite-dimensional vectors) that optimize a functional. A functional is a mapping from a space of functions to real numbers: $L:{\mathbb{R}}^{\infty }\to \mathbb{R}$ (technically, $H^1\to \mathbb{R}$).

Concept of Functional and Variational Calculus

• In classical (discrete) optimisation, we minimize a function of $D$ variables
• In variational calculus, we consider a continuum of variables $y(x)$, where $x$ is continuous – it can be seen as a limit $D\to \infty$
• This approach allows us to optimize over entire functions rather than discrete parameters

Variation of a Functional

For a function $y(x)$ and a “direction” $\delta y(x)$ (which is another function), the variation of objective function is:

$\delta L(y;\delta y):=L(y+\epsilon \delta y)-L(y)$

The “derivative” of this variation should be zero for all admissible $\delta y$:

${\lim }_{\epsilon \to 0}\frac{\delta L(y;\delta y)}{\epsilon }=0\forall \text{ admissible }\delta y(x)$

Euler-Lagrange Equation

For a functional of the form:

$L(y)={\int }_a^{b}f(y,y^{\prime },x)dx$

where $y^{\prime }$ represents $\frac{dy}{dx}$, the necessary condition for an extremum is given by the Euler-Lagrange equation:

$\frac{\partial f}{\partial y}-\frac{d}{dx}\left(\frac{\partial f}{\partial y^{\prime }}\right)=0$

Derivation of Euler-Lagrange Equation

Starting with: $\delta L(y,y^{\prime },x;\delta y,\delta y^{\prime })={\int }_A^B\left[\frac{\partial f}{\partial y}\epsilon \delta y+\frac{\partial f}{\partial y^{\prime }}\epsilon \delta y^{\prime }\right]dx$

Using integration by parts for the second term: ${\int }_A^B\frac{\partial f}{\partial y^{\prime }}\epsilon \delta y^{\prime }dx={\left[\frac{\partial f}{\partial y^{\prime }}\epsilon \delta y\right]}_A^B-{\int }_A^B\frac{d}{dx}\left(\frac{\partial f}{\partial y^{\prime }}\right)\epsilon \delta ydx$

For admissible $\delta y$, we have $\delta y(x_A)=\delta y(x_B)=0$, so: $\delta L(y,y^{\prime },x;\delta y,\delta y^{\prime })={\int }_A^B\left[\frac{\partial f}{\partial y}-\frac{d}{dx}\left(\frac{\partial f}{\partial y^{\prime }}\right)\right]\epsilon \delta ydx$

Since this must be zero for all $\delta y$, we get the Euler-Lagrange equation.

Example Problems

The Brachistochrone Problem

Problem Statement: Given two points A and B, find the path joining these points such that a bead sliding down under gravity travels from A to B in the shortest time.

Objective Function: $L(y)={\int }_A^B\sqrt{\frac{1+(dy/dx{)}^2}{2gy}}dx$

This is a classic problem that can be solved using the Euler-Lagrange equation. The solution is a cycloid.

Minimum Distance Path

Objective Function: $L(y)={\int }_A^B\sqrt{1+y^{\prime 2}}dx$

Here:

• $f(y,y^{\prime },x)=\sqrt{1+y^{\prime 2}}$
• $\partial f/\partial y=0$
• $\partial f/\partial y^{\prime }=\frac{y^{\prime }}{\sqrt{1+y^{\prime 2}}}$

Applying the Euler-Lagrange equation yields $y^{\prime }=\beta$ (constant), so the solution is a straight line.

Connection to Other Areas

1. Finite element method (FEM) is based on variational calculus (minimizing total system energy)
2. Lagrange multiplier and KKT conditions extend to the continuum limit of variational calculus
3. Contact and plasticity in FEM (constraints) also lead to KKT conditions
4. Many physics problems can be formulated as minimization of energy functionals (principle of least action)

Key Differences from Discrete Optimization

1. Instead of algebraic equations, we get differential equations as the first-order necessary conditions
2. These differential equations, along with boundary conditions, need to be solved to find the extrema
3. The solution is a function rather than a finite set of parameters
4. Numerical solutions typically involve discretization (returning to finite dimensions)