Fourier Analysis For Optimization Problems

The above analysis using Fourier decomposition can also serve for the analysis of optimization problems. Of probably the main concern for us is to define optimization problems for our engineering tasks, that will be mathematically "good", or well-posed in mathematical terminology. We would like the problem to have a solution (existence), that the solution will be unique (uniqueness) and that the solution will depend in a continuous way on other parameters in the problem (continuous dependence). We will see that these properties of the problem, at least when considering the high frequency range, can be easily analyzed. The usual rigorous mathematical techniques for these question are very complex and may not be of a practical engineering use. Moreover, some important details which are of engineering importance are not present in the rigorous analysis, while they are present in the formal Fourier techniques.

The characterization of the minimizer for an optimization problem gives the equation

$$\nabla E (\alpha ^*) = 0.$$ (33)

This is in general a nonlinear equation for the unknown $\alpha$. Now lets say that we have an approximate solution $\alpha$ and we are seeking the correction $\tilde\alpha$ such that $\nabla E ( \alpha + \tilde\alpha ) = 0$. Using a Taylor expansion we see that $\tilde\alpha$ satisfies approximately the equation

$${\cal H} \tilde \alpha = -\nabla E (\alpha)$$ (34)

where ${\cal H}$ is the Hessian of the functional.

If the design variable $\alpha$ can be decomposed in a Fourier series then important information can be obtain about the problem using Fourier analysis.

The symbol of the Hessian, $\hat{\cal H}(\bf k)$ contains all of the necessary information for analyzing and designing optimization procedures. Notice that, at the vicinity of the minimum, the gradient of the functional, $g$, is linearly related to the error. In the Fourier space the relation is given by

$$\hat {\cal H} ( {\bf k} ) \hat{\tilde\alpha} ({\bf k}) = - \hat g ( {\bf k})$$ (35)

where $\hat{\tilde\alpha} ({\bf k})$ and $\hat g ( {\bf k})$ are the Fourier transforms of $\tilde \alpha, g$, respectively.