Engineering Optimisation Tutorial 1

For each of the problem below, • Identify the loss function L, design variable θ, and its dimension D • Use Python/calculator to plot the loss function versus design variable in the given range • Is the loss function continuous or discontinuous or discrete? • Visually, does the loss function look convex? • Visually, is there a minimum or a maximum? Are these local or global? • Derive the necessary and sufficient condition (up to second order derivative) for minimum. No need to solve them, though. • Is the loss function linear/quadratic/nonlinear? What effect does it have on the necessary/sufficient conditions?

Question 1

Projectile range: One can throw a rock at a velocity of 10 m/s at any angle. On a flat surface, the distance traveled by projectile is

d = \frac{2v_o^2}g\cos(\theta)\sin(\theta) $$where $v_0$ is the velocity of the projectile at angle $θ$, and $g$ is the gravitational acceleration. Plot between θ = 0 and 90 degrees. - **Identify the loss function L, design variable θ, and its dimension D** This is actually maximisation, so we're doing the loss inverse to normal so:

\mathcal L(\theta) = -d, \quad \theta(\text{design variable})=\theta(\angle), \quad D=1

- **Is the loss function continuous or discontinuous or discrete?** its essentially $\cos(\theta)\sin(\theta)$, so its continuous - **Visually, does the loss function look convex?** No? - **Visually, is there a minimum or a maximum? Are these local or global?** Yeah for this we should be getting global maxima at $45^o$ - **Derive the necessary and sufficient condition (up to second order derivative) for minimum. No need to solve them, though.** Necessary Condition: $\frac{\partial \mathcal L}{\partial \theta} = 0$ So:

\frac{\partial}{\partial\theta}[ -\frac{2v_o^2}g\cos(\theta)\sin(\theta)] = 0

-\frac{2v_o^2}g(\cos(\theta)\cos(\theta))-(\sin(\theta)\sin(\theta)) = 0

-\frac{2v_o^2}g\cos(2\theta) = 0

The sufficient condition therefore is $\frac{\partial \mathcal L^2}{\partial \theta^2} > 0$

\frac{4v_o^2}g\sin(2\theta) > 0

- **Is the loss function linear/quadratic/nonlinear? What effect does it have on the necessary/sufficient conditions?** Non-linear, its trigonometric ??? ## Question 2 **Beam design**: We want to design a cantilever 1 meter-long beam that can support a vertical load of 10 N at its free end. If the cross-section is circular, what should be its diameter to minimize the deflection of the beam? Maximum deflection of the beam (from structural mechanics) is given by

\delta = \frac{64FL^3}{3\pi ED^4}

$$whereFistheappliedforce,Listhelengthofthebeam,E=1GPaistheYoung’smodulus,andDisthediameterofthebeam.Plotfordiameterbetween1cmand5cm.•\ast \ast IdentifythelossfunctionL,designvariable\theta ,anditsdimensionD\ast \ast$$

\mathcal L(D) = \delta = \frac{64FL^3}{3\pi ED^4}

The design variable is D, the diameter. The Dimension is D=1 •**Is the loss function continuous or discontinuous or discrete?** Continous? • **Visually, does the loss function look convex?** Between 1 and 5, and $D^4$, shouldnt be convex • **Visually, is there a minimum or a maximum? Are these local or global?** Should be a minimum at $\max D$ as $\frac{c}{cD^4}$ so local minimum • **Derive the necessary and sufficient condition (up to second order derivative) for minimum. No need to solve them, though.** Necessary : $\mathcal L'(D) = 0$

\frac{64FL^3}{3\pi E} * \frac{\partial}{\partial D}[D^{-4}] = -\frac{256FL^3}{3\pi ED^5} = 0

Sufficient : $\mathcal L ''(D) > 0$

\frac{1280FL^3}{3\pi ED^6} > 0

F,L,E all positive, and D > 0, therefore if we have a critical point, its a minimum • **Is the loss function linear/quadratic/nonlinear? What effect does it have on the necessary/sufficient conditions?** Non-linear ($D^4$) ## Question 3 Inventory problem: find the order size Q to minimise the cost per year

Cost = \frac{\lambda K}{Q} + \lambda c + \frac{hQ}2

where λ = 200 units/year is the demand rate, K = 100 £ is the fixed cost per order placement, c = 20 £ is the acquisition cost per unit, and h = 4 £ per unit per year is the storage cost. Plot for Q between 10 and 1000. • **Identify the loss function L, design variable θ, and its dimension D** $\mathcal L (Q) = \frac{\lambda K}{Q} + \lambda c + \frac{hQ}2$ $\theta = Q$ $D=1$ •**Is the loss function continuous or discontinuous or discrete?** Continous between 10-1000 • **Visually, does the loss function look convex?** Convex (single min) • **Visually, is there a minimum or a maximum? Are these local or global?** Global Min • **Derive the necessary and sufficient condition (up to second order derivative) for minimum. No need to solve them, though.** Necessary : $\mathcal L '(Q) = -\frac{\lambda K }{Q^2} + \frac h2$ = 0 Sufficient : $\mathcal L ''(Q) = \frac{2\lambda K}{Q^3} > 0$ $\lambda$, K, Q are always positive, therefore we have a minimum. • **Is the loss function linear/quadratic/nonlinear? What effect does it have on the necessary/sufficient conditions?** Non-linear (Q in denominator) ## Question 4

x(t) = e^{\xi\omega_n t}\frac{v_0}{\omega_d}sin(\omega_dt)

where ξ = 0.1 is the damping ratio, ωn = 1 rad/sec is the natural frequency, t is the time, v0 = −1 cm/s is the initial velocity, and ωd = ωn p 1 − ξ 2. What is the minimum displacement? Plot for time between 0 and 50 seconds. • **Identify the loss function L, design variable θ, and its dimension D** $\mathcal L(t) = e^{\xi\omega_n t}\frac{v_0}{\omega_d}sin(\omega_dt)$ $\theta = t$ $D=1$ •**Is the loss function continuous or discontinuous or discrete?** Continous • **Visually, does the loss function look convex?** No, oscillates • **Visually, is there a minimum or a maximum? Are these local or global?** minimum, decreasing oscillations, global • **Derive the necessary and sufficient condition (up to second order derivative) for minimum. No need to solve them, though.** $u= e^{\xi\omega_nt}, v = \sin(\omega_dt)$

\mathcal L’(t) = \frac{v_o}{\omega_d}[uv’+vu’] = \frac{v_o}{\omega_d}[\omega_d\cos(\omega_dt)e^{\xi\omega_nt}+\xi\omega_n\sin(\omega_dt)e^{\xi\omega_nt}] = 0

\mathcal L”(t) = \frac{v_o}{\omega_d}[\omega_d(-\omega_d\sin(\omega_dt)e^{\xi\omega_nt} + \xi\omega_n\cos(\omega_dt)e^{\xi\omega_nt})+\xi\omega_n(\omega_d\cos(\omega_dt)e^{\xi\omega_nt} + \xi\omega_n\sin(\omega_dt)e^{\xi\omega_nt})] > 0

\mathcal L''(t) = v_oe^{\xi\omega_nt}\left[\left(\frac{\xi^2\omega^2_n}{\omega_d}-\omega_d \right) + \sin(\omega_dt) + 2\xi\omega_n\cos(\omega_dt) \right] >0

• **Is the loss function linear/quadratic/nonlinear? What effect does it have on the necessary/sufficient conditions?** non linear, trig, and exp ## Question 5 Energy minimisation: Total energy of a spring under external force is given by:

E = \frac12kx

where k = 100 N/m is the spring stiffness, F = 1 N is the applied force, and x is the change in length of the spring. What is the minimum total energy of the system? Plot for displacement between 0 and 2 cm. • **Identify the loss function L, design variable θ, and its dimension D** $\mathcal L (x) = \frac12kx^2 - Fx$ $\theta = x$ $D = 1$ •**Is the loss function continuous or discontinuous or discrete?** Continous • **Visually, does the loss function look convex?** Yes, looks like a basic quadratic with a min • **Visually, is there a minimum or a maximum? Are these local or global?** global min • **Derive the necessary and sufficient condition (up to second order derivative) for minimum. No need to solve them, though.** $\mathcal L'(x) = kx -F = 0$ $L''(x) = k > 0$ • **Is the loss function linear/quadratic/nonlinear? What effect does it have on the necessary/sufficient conditions?** Quadratic. ## Question 6 Distance traveled: Distance traveled in time t at velocity v is given by $$d = vt$$. What is the maximum distance one can travel in 1 hour? Plot for velocity between 0 and 100 m/s. • **Identify the loss function L, design variable θ, and its dimension D** $\mathcal L(v) = -vt$ $\theta = v$ $D=1$ •**Is the loss function continuous or discontinuous or discrete?** Continous • **Visually, does the loss function look convex?** No, linear • **Visually, is there a minimum or a maximum? Are these local or global?** No • **Derive the necessary and sufficient condition (up to second order derivative) for minimum. No need to solve them, though.** $\mathcal L'(v) = -t = 0$ $\mathcal L ''(v) = 0 > 0$ This can't be true, no solution For us then, we just pick the largest v • **Is the loss function linear/quadratic/nonlinear? What effect does it have on the necessary/sufficient conditions?** Linear, this means that there is no sufficient, and # Section 2 For the minimisation problems described above, perform 3 iterations using steepest gradient (∆θ = −gi) and Newton–Raphson methods (∆θ = −gi/Hi) with the starting guess as given below (a) Inventory problem: starting guess of 50 (b) Mass-spring oscillation: starting guess of 1 second (c) Mass-spring oscillation: starting guess of 5 seconds ## Question 1

Cost = \frac{\lambda K}{Q} + \lambda c + \frac{hQ}2

$\mathcal L '(Q) = -\frac{\lambda K}{Q^2} + \frac h2$ $L''(Q) = \frac{2\lambda K}{Q^3}$ **Steepest Grad** At $\theta_0$ = 50 L($\theta$) = 4500, $g_0$=-6, $\Delta \theta$ = -$g_0$ = 6, $\theta_1$ = 56 At $\theta_0$ = 56 L($\theta$) = 4469.14 , $g_0$=-4.378, $\Delta \theta$ = -$g_0$ = 4378, $\theta_1$ = 60.378 $\theta_0$ = 60.378 L($\theta$) = 4452, $g_0$=-3.486, $\Delta \theta$ = -$g_0$ = 3.486, $\theta_1$ = 63.86 **Newton Raphson**

\begin{array}{ll} \text { At } \theta_0=50 \ \mathrm{L}\left(\theta_0\right)=4500, \quad \mathrm{~g}_0=\mathrm{L}^{\prime}\left(\theta_0\right)=-6 ,\quad\theta_1=\theta_0+\Delta \theta=68.75 \end{array}

\begin{array}{ll} \text { At } \theta_1=68.75 \ L\left(\theta_1\right)=4428.4, g_1=L^{\prime}\left(\theta_1\right)=-2.23 & , H_1=L^{\prime \prime}\left(\theta_1\right)=0.123 \quad, \Delta \theta=-g_1 / H_1=18.13, \quad \theta_2=\theta_1+\Delta \theta=86.88 \end{array}

At $\theta_2=86.88$

\begin{aligned} & L\left(\theta_2\right)=4403.96, g_2=L^{\prime}\left(\theta_2\right)=-0.65 \quad, H_2=L^{\prime \prime}\left(\theta_2\right)=0.06 \quad, \Delta \theta=-g_2 / H_2=10.65, \quad \theta_3=\theta_2+\Delta \theta=97.53 \end{aligned}

## Question 2 The gradient is:

g = (v_0/ω_d) × e^{-ξωnt} × [(-ξωn)\sin(ω_dt) + ω_d\cos(ω_dt)]

|Iteration|t|x(t)|Gradient g|Step Δt = -g × 0.5|New t| |---|---|---|---|---|---| |0|1.00|-0.8679|0.4540|-0.2270|0.7730| |1|0.7730|-0.7112|0.6784|-0.3392|0.4338| |2|0.4338|-0.4291|0.8951|-0.4476|-0.0138| The steepest gradient method is oscillating around the minimum point. #### Newton-Raphson Method: Δθ = -gi/Hi

H_i = \mathcal L''(t) = v_oe^{\xi\omega_nt}\left[\left(\frac{\xi^2\omega^2_n}{\omega_d}-\omega_d \right) + \sin(\omega_dt) + 2\xi\omega_n\cos(\omega_dt) \right] >0

| Iteration | t | x(t) | Gradient g | Hessian H | Step Δt = -g/H | New t | | --------- | ------- | ------- | ---------- | --------- | -------------- | ------- | | 0 | 1.00 | -0.8679 | 0.4540 | 0.8613 | -0.5271 | 0.4729 | | 1 | 0.4729 | -0.4644 | 0.8721 | 0.4901 | -1.7795 | -1.3066 | | 2 | -1.3066 | 0.9649 | 0.2521 | -0.9663 | 0.2609 | -1.0457 | The Newton-Raphson method is also showing oscillatory behavior due to the complexity of the function. ## Question 3 #### Steepest Gradient Method: Δθ = -gi $g = (v_0/ω_d) × e^{-ξωnt} × [(-ξωn)\sin(ω_dt) + ω_d\cos(ω_dt)]$ |Iteration|t|x(t)|Gradient g|Step Δt = -g × 0.5|New t| |---|---|---|---|---|---| |0|5.00|0.3662|-0.3604|0.1802|5.1802| |1|5.1802|0.3264|-0.3896|0.1948|5.3750| |2|5.3750|0.2764|-0.4157|0.2079|5.5829| Starting from t = 5 seconds, the method is moving toward a local minimum, but convergence is slow. #### Newton-Raphson Method: Δθ = -gi/Hi

H_i = \mathcal L''(t) = v_oe^{\xi\omega_nt}\left[\left(\frac{\xi^2\omega^2_n}{\omega_d}-\omega_d \right) + \sin(\omega_dt) + 2\xi\omega_n\cos(\omega_dt) \right] >0

|Iteration|t|x(t)|Gradient g|Hessian H|Step Δt = -g/H|New t| |---|---|---|---|---|---|---| |0|5.00|0.3662|-0.3604|-0.3674|0.9809|5.9809| |1|5.9809|0.1276|-0.4612|0.1254|-3.6780|2.3029| |2|2.3029|-0.2103|0.4572|0.2043|-2.2379|0.0650| ## Section 3 Neccesary condition at $\mathcal L '(\theta) = 0, \quad 3\theta^2=0$ , this is satisified at $\theta$ = 0 The sufficient conditions would be $6\theta > 0$ for min, or $6\theta < 0$ for max. But at $\theta =0$ we get neither, so a saddle/inflection point ## Section 4 Consider the quadratic objective function:

f(\theta) = a\theta^2 + b\theta + c

Start with an initial guess $\theta_0$. The first derivative is:

f’(\theta) = 2a\theta + b

$$Thesecondderivativeis:$$

f”(\theta) = 2a

$$---\ast \ast \ast NewtonUpdateIteration1:\ast \ast \ast ThefirstNewtonupdatestepis:$$

\Delta \theta = -\frac{f’(\theta_0)}{f”(\theta_0)} = -\frac{2a\theta_0 + b}{2a}

Therefore, the updated guess $\theta_1$ is:

\theta_1 = \theta_0 + \Delta \theta = \theta_0 - \frac{2a\theta_0 + b}{2a} = -\frac{b}{2a}

*Note: This result is independent of the initial guess $\theta_0$.* --- ***Evaluate derivatives at $\theta_1 = -\frac{b}{2a}$:*** - **First derivative**:

f’(\theta_1) = 2a\left(-\frac{b}{2a}\right) + b = -b + b = 0

$$-\ast \ast Secondderivative\ast \ast :$$

f”(\theta_1) = 2a

$$---\ast \ast \ast NewtonUpdateIteration2:\ast \ast \ast ThesecondNewtonupdatestepis:$$

\Delta \theta = -\frac{f’(\theta_1)}{f”(\theta_1)} = -\frac{0}{2a} = 0

Thus, the second step $\Delta \theta$ is zero, demonstrating convergence after just one iteration for a quadratic function.