Introduction

In this lecture we discuss the use of elementary theory of pseudo-differential operators to analyze Hessians for optimization problems governed by partial differential equations. The role of this analysis is to come up with Hessian approximation that will accelerate gradient based methods. Quasi-Newton methods such as BFGS cannot cope efficiently with a very large dimension of the design space. Their efficiency is very good for a small dimensional design space, and it deteriorate as the dimension of the design space increases. The approach presented here is a complementing approach for the classical quasi-Newton methods. It uses the asymptotic behavior of the symbol of the Hessian to construct an accurate Hessian approximation for the high frequency range, in the representation of the design space. The resulting approximate Hessians are differential or pseudo-differential operators.