What is Optimisation?
Optimisation is the process of finding the best solution (minimum or maximum) for a given problem under specified constraints. In engineering, we frequently need to determine optimal designs, configurations, or control parameters that maximize efficiency, minimize cost, or achieve other performance objectives.
Applications in Engineering
Optimisation is crucial in many engineering disciplines:
- Mechanical Engineering: Minimizing weight while maintaining strength
- Civil Engineering: Optimizing structural designs
- Electrical Engineering: Maximizing signal-to-noise ratio
- Chemical Engineering: Optimizing reaction rates or yields
- Industrial Engineering: Minimizing operational costs
Key Characteristics of Optimisation
Iterative Nature
Optimisation is inherently iterative. We typically:
- Start with an initial guess
- Evaluate the performance
- Update the guess in a direction that improves performance
- Repeat until convergence
Most problems cannot be solved in a single step and require multiple iterations to reach an optimal solution.
Model-Based vs. Black-Box Optimisation
- Model-based optimisation: Uses mathematical models to predict system behavior. More efficient when accurate models are available.
- Black-box optimisation: Treats the system as a black box, only observing inputs and outputs without internal knowledge. Useful when models are unavailable or complex.
Basic Components of an Optimisation Problem
- Objective Function: The mathematical expression to be minimized or maximized
- Design Variables: The parameters that can be adjusted to optimize the objective
- Constraints: Limitations on the design variables or outputs
- Search Space: The range of possible values for the design variables
Mathematical Representation
For a minimization problem:
- Objective function: L(θ)
- Design vector: θ ∈ ℝᴰ (D-dimensional)
- Optimal solution: θ* = argmin L(θ)
For a maximization problem:
- Can be converted to minimization: θ* = argmax L(θ) = argmin [-L(θ)]
Example: Simple Optimisation Problem
Consider a project to maximize the range of a projectile launched from a fixed height:
- Objective: Maximize distance traveled
- Design variable: Launch angle θ
- Constraints: Fixed initial velocity, fixed launch height
- Model: Projectile motion equations from physics
This is an example of a single-variable (univariate) optimisation problem where we seek the optimal angle that maximizes the range.