The Symbol of The Hessian

In order to get a quantitative description of the level curves of the cost functional and to be able to determine the structure of the functional near the minimum, a Fourier analysis of the Hessian of the functional is carried out. In many problems of engineering interest the design variable are associated with boundary quantities and the gradient of the cost function as well as the Hessian are quantities defined on part of the boundary as well.

We have seen in lecture 1 that the eigenvalue distribution of the Hessian plays an important role in convergence rates for the optimization problem. Moreover, the asymptotic behavior of the large eigenvalues of the Hessian is tightly related to the symbol of the Hessian. Its computation is therefore of practical importance.

In the next example we calculate the symbol of the Hessian for a control problem related to a shape design problem. We identify in this case the Hessian as a differential operator acting on functions defined on the boundary of the domain.

Example V Consider the following minimization problem,

$$\min_{\alpha} \frac{1}{2} \int _{\partial\Omega} (u - u^*)^2 dx$$ (36)

subject to

$$\left( \begin{array}{cr} \beta^2 \frac{\partial }{\partial x}& \f... ... u \\ v \end{array}\right)= \left( \begin{array}{c} 0 \\ 0 \end{array}\right)$$ (37)

with the boundary condition

$$v = \frac{\partial \alpha}{\partial x} \qquad \mbox{\rm$\partial\Omega$}$$ (38)

where $\beta^2 = (1 - M^2)$ and $\Omega =\{(x,y)\vert y > 0\}$. We introduce adjoint variables (Lagrange multipliers) $(\lambda, \mu)$ which can be shown to satisfy

$$\left( \begin{array}{cr} \beta^2 \frac{\partial }{\partial x}& \f... ... \\ \mu \end{array}\right)= \left( \begin{array}{c} 0 \\ 0 \end{array}\right)$$ (39)

with the boundary condition

$$-\mu + (u-u^*) = 0 \qquad \qquad \partial\Omega .$$ (40)

At the minimum the following equation has to be satisfied

$$\frac{\partial \lambda }{\partial x} = 0 \qquad \qquad \partial \Omega .$$ (41)

The left hand side of this equation is the gradient of the functional subject to the PDE, and its behavior in the vicinity of the minimum needs to be analyzed. In order to do that we examine the perturbation in the solution as a result of a perturbation in the design function $\alpha$. The linearity of the interior equation implies that the perturbation variables $\tilde{u}, \tilde{v}, \tilde{\lambda} , \tilde{\mu}$ satisfy the same equations as $u,v,\lambda,\mu$ in the interior of the domain, and the boundary conditions for them are

$$\begin{array}{lll} \tilde{v} - \frac{\partial \tilde{\alpha}}{\pa... ...rtial\Omega \\ \tilde u -\tilde{\mu} & = 0 & \qquad \partial\Omega \end{array}$$ (42)

and the change in the gradient is given by

$$\frac{\partial \tilde \lambda}{\partial x}_{\vert _{\partial\Omega}} .$$ (43)

The analysis continues by assuming $\tilde\alpha$ to be a Fourier component, that is,

$$\tilde \alpha = \exp ( i k x).$$ (44)

From the boundary condition we get $\tilde v _{\vert _{\partial\Omega}} = i k \exp( i k x)$ and using the interior equations (37) for $(\tilde u, \tilde v)$ we conclude that

$$\left( \begin{array}{c} \tilde u \\ \tilde v \end{array}\right) ... ...vert /\beta \\ i k \end{array}\right) e^{ i k x} e^{- \beta \vert k \vert y }.$$ (45)

Using the boundary conditions for $\tilde \mu$ we get $\tilde \mu _{\vert _{\partial\Omega}} = \vert k \vert / \beta \exp ( i k x)$ and from the interior equations (39) for $(\tilde \lambda, \tilde \mu)$ it is easy to see that

$$\left( \begin{array}{c} \tilde \lambda \\ \tilde \mu \end{array}... ... \vert k\vert /\beta \end{array}\right) e^{i k x} e^{ - \beta \vert k \vert y }$$

Combining these results we obtain that the change in the gradient, corresponding to a change in the design variable by $\tilde \alpha = \exp (i k x )$ is

$$\frac{\partial \tilde \lambda } { \partial x}_{\vert _{\partial\Omega}} = \frac{\vert k\vert^2}{\beta ^2} \tilde \alpha$$ (46)

Thus, the symbol of the Hessian $\hat{\cal H} (k)$ is given by

$$\hat{\cal H} (k ) = \frac{\vert k\vert^2}{\beta ^2}.$$ (47)

This symbol correspond to the differential operator

$${\cal H} = - \frac{1}{\beta ^2} \frac{d^2}{dx^2}.$$ (48)

A General Remark: This analysis was performed on problems with constant coefficients, however, it is not limited to such cases. It can be applied to non-constant coefficients and nonlinear problems in general domains. In such cases one linearizes the problem (if it is nonlinear) and freezes coefficients at a point $x_0$ in the domain. A constant coefficient problem is obtained which describes the behavior of the problem in a small vicinity of that point. The validity of the resulting Fourier analysis for that problem is then restricted to a small vicinity. Usually, all expressions in the analysis will depend on the frozen coefficients and analyzing one point in the domain is enough to obtain the desired information about all points. The rigorous justification of this process is beyond the scope of our discussion here.

When considering the problem in a general domain one perform the analysis at a boundary point by transforming the vicinity of that point into 'half-space' and applying there the analysis presented above. Thus, smooth boundaries can be treated. This analysis is local and hence is relevant for high frequencies only. Low frequencies are affected by the shape of the boundary and cannot be analyzed using local techniques. However, in spite of this limitation, it is still a very useful tool for quantitative results regarding our problems.