Skip to main content Accessibility help

A Fast Solver for an H1 Regularized PDE-Constrained Optimization Problem

  • Andrew T. Barker (a1), Tyrone Rees (a2) and Martin Stoll (a3)

In this paper we consider PDE-constrained optimization problems which incorporate an H1 regularization control term. We focus on a time-dependent PDE, and consider both distributed and boundary control. The problems we consider include bound constraints on the state, and we use a Moreau-Yosida penalty function to handle this. We propose Krylov solvers and Schur complement preconditioning strategies for the different problems and illustrate their performance with numerical examples.

Corresponding author
*Corresponding author. Email (A. T. Barker), (T. Rees), (M. Stoll)
Hide All
[1]Zhong-Zhi, Bai: Block preconditioners for elliptic PDE-constrained optimization problems. Computing 91(4), 379395 (2011)
[2]Bangerth, W., Hartmann, R., Kanschat, G.: deal. II—a general-purpose object-oriented finite element library. ACM Trans. Math. Software 33(4), Art. 24, 27 (2007)
[3]Barrett, R., Berry, M., Chan, T. F., Demmel, J., Donato, J., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., and der Vorst, H. V.: Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods. 2nd Edition, SIAM, Philadelphia, PA, 1994.
[4]Benzi, M., Golub, G.H., Liesen, J.: Numerical solution of saddle point problems. Acta Numer 14, 1137 (2005)
[5]Benzi, M., Haber, E., Taralli, L.: A preconditioning technique for a class of PDE-constrained optimization problems. Advances in Computational Mathematics 35, 149173 (2011)
[6]Bergounioux, M., Ito, K., Kunisch, K.: Primal-dual strategy for constrained optimal control problems. SIAM J. Control Optim. 37(4), 11761194 (1999)
[7]Bergounioux, M., Kunisch, K.:Primal-dual strategy for state-constrained optimal control problems, Comput. Optim. Appl. 22(2), 193224 (2002) DOI 10.1023/A:1015489608037
[8]Bochev, P., Lehoucq, R.: On the finite element solution of the pure neumann problem. SIAM review 47(1), 5066 (2005)
[9]Bramble, J.H., Pasciak, J.E.: A preconditioning technique for indefinite systems resulting from mixed approximations of elliptic problems. Math. Comp 50(181), 117 (1988)
[10]Cai, Xiao-Chuan and Liu, Si and Zou, Jun: An overlapping domain decomposition method for parameter identification problems. Domain Decomposition Methods in Science and Engineering XVII, 60, 451458 (2008)
[11]Casas, E.: Control of an elliptic problem with pointwise state constraints. SIAM J. Control Optim. 24(6), 13091318 (1986). DOI 10.1137/0324078. URL
[12]Casas, E., Herzog, R. and Wachsmuth, G.: Approximation of sparse controls in semilinear equations by piecewise linear functions. Numerische Mathematik 122, 645669 (2012)
[13]Chan, R.H. and Chan, T.F. and Wan, WL and others: Multigrid for differential-convolution problems arising from image processing. Proc. Workshop on Scientific Computing, pp. 5872 (1997)
[14]Chan, T.F. and Tai, X.C.: Identification of discontinuous coefficients in elliptic problems using total variation regularization. SIAM Journal on Scientific Computing 25(3) 881904 (2003)
[15]Choi, Y., Farhat, C., Murray, W. and Saunders, M.: A practical factorization of a Schur complement for PDE-constrained Distributed Optimal Control. arXiv preprint arXiv:1312.5653 (2013)
[16]Christofides, P.: Nonlinear and robust control of PDE systems: Methods and applications to transport-reaction processes. Birkhauser (2001)
[17]Cimrák, I. and Melicher, V.: Mixed Tikhonov regularization in Banach spaces based on domain decomposition submitted to Applied Mathematics and Computation (2012)
[18]Collis, S.S., Ghayour, K., Heinkenschloss, M., Ulbrich, M. and Ulbrich, S.: Numerical solution of optimal control problems governed by the compressible Navier-Stokes equations International series of numerical mathematics, 4356 (2002)
[19]van den Doel, K. and Ascher, U. and Haber, E.: The lost honour of l2-based regularization Submitted (2012)
[20]Du, X., Sarkis, M., Schaerer, C.E., Szyld, D.B.: Inexact and truncated parareal-in-time Krylov subspace methods for parabolic optimal control problems. Tech. Rep. 12-02-06, Department of Mathematics, Temple University (2012)
[21]Duff, I.S., Erisman, A.M., Reid, J.K.: Direct methods for sparse matrices. Monographs on Numerical Analysis. The Clarendon Press Oxford University Press, New York (1989)
[22]Duff, Iain S.: MA57—a code for the solution of sparse symmetric definite and indefinite systems. ACM Transactions on Mathematical Software (TOMS) 30(2) 118144 (2004)
[23]Elman, H.C., Silvester, D.J., Wathen, A.J.: Finite elements and fast iterative solvers: with applications in incompressible fluid dynamics. Numerical Mathematics and Scientific Computation. Oxford University Press, New York (2005)
[24]Falgout, R.: An Introduction to Algebraic Multigrid. Computing in Science and Engineering, 8 (2006), pp. 2433. Special Issue on Multigrid Computing.
[25]Fletcher, R.: Conjugate gradient methods for indefinite systems, Lecture Notes in Mathematics, vol. 506. Springer-Verlag, Berlin (1976), 7389.
[26]Freund, R.W., Nachtigal, N.M.: QMR: a quasi-minimal residual method for non-Hermitian linear systems. Num. Math. 60(1) 315339 (1991)
[27]Gee, M., Siefert, C., Hu, J., Tuminaro, R., Sala, M.: ML 5.0 smoothed aggregation user's guide. Tech. Rep. SAND2006-2649, Sandia National Laboratories (2006)
[28]Gunzburger, Max D.: Perspectives in flow control and optimization, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA (2003)
[29]Greenbaum, A.: Iterative methods for solving linear systems, Frontiers in Applied Mathematics, vol. 17. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA (1997)
[30]Griewank, A., Walther, A.: Algorithm 799: revolve: an implementation of checkpointing for the reverse or adjoint mode of computational differentiation. ACM Transactions on Mathematical Software (TOMS) 26(1), 1945 (2000)
[31]Haber, E. and Hanson, L.: Model Problems in PDE-Constrained Optimization Emory University TR-2007-009 (2007)
[32]Hackbusch, W.:Multigrid methods and applications. vol. 4 of Springer Series in Computational Mathematics, Springer-Verlag, Berlin, 1985.
[33]Heinkenschloss, M.: A time-domain decomposition iterative method for the solution of distributed linear quadratic optimal control problems. J. Comput. Appl. Math. 173(1), 169198 (2005). DOI 10.1016/
[34]Heinkenschloss, M.: Formulation and analysis of a sequential quadratic programming method for the optimal Dirichlet boundary control of Navier-Stokes flow Optimal Control, Theory, Algorithms, and Applications (1998)
[35]Heinkenschloss, Matthias, Ridzal, Denis: A matrix-free trust-region SQP method for equality constrained optimization. Technical Report 11-17, Department of Computational and Applied Mathematics, Rice University, Houston, Texas, 2011.
[36]Herzog, R., Sachs, E.W.: Preconditioned conjugate gradient method for optimal control problems with control and state constraints. SIAM J. Matrix Anal. Appl. 31(5), 22912317 (2010)
[37]Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stand 49, 409436 (1953) (1952)
[38]Hintermüller, M., Ito, K., Kunisch, K.: The primal-dual active set strategy as a semismooth Newton method. SIAM J. Optim. 13(3), 865888 (2002)
[39]Hintermüller, M., Kunisch, K.: Path-following methods for a class of constrained minimization problems in function space. SIAM J. Optim. 17(1), 159187 (2006)
[40]Hinze, M., Köster, M., Turek, S.: A Hierarchical Space-Time Solver for Distributed Control of the Stokes Equation. Tech. rep., SPP1253-16-01 (2008)
[41]Hinze, M., Pinnau, R., Ulbrich, M., Ulbrich, S.: Optimization with PDE Constraints. Mathematical Modelling: Theory and Applications. Springer-Verlag, New York (2009)
[42]Hogg, Jonathan D., and Scott, Jennifer A.. HSL_MA97: A bit-compatible multifrontal code for sparse symmetric systems. Science and Technology Facilities Council, 2011.
[43]Ito, K., Kunisch, K.: Semi-smooth Newton methods for state-constrained optimal control problems. Systems Control Lett. 50(3), 221228 (2003). DOI 10.1016/S0167-6911(03)00156-7
[44]Ito, K., Kunisch, K.: Lagrange multiplier approach to variational problems and applications, Advances in Design and Control, vol. 15. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA (2008)
[45]Kanzow, C.: Inexact semismooth Newton methods for large-scale complementarity problems. Optimization Methods and Software 19(3-4), 309325 (2004). DOI 10.1080/10556780310001636369. URL
[46]Keung, Y.L. and Zou, J.: Numerical identifications of parameters in parabolic systems Inverse Problems 14(1), 83100 (1999)
[47]Kollmann, M., Kolmbauer, M.: A Preconditioned MinRes Solver for Time-Periodic Parabolic Optimal Control Problems. Sumitted, Numa-Report 2011-06 (August 2011)
[48]Li, F. and Shen, C. and Li, C.: Multiphase Soft Segmentation with Total Variation and H1 Regularization Journal of Mathematical Imaging and Vision 37(2), 98111 (2010)
[49]Ng, M.K. and Chan, R.H. and Chan, T.F. and Yip, A.M.: Cosine transform preconditioners for high resolution image reconstruction Linear Algebra and its Applications 316(1) 89104 (2000)
[50]Murphy, M.F., Golub, G.H., Wathen, A.J.: A note on preconditioning for indefinite linear systems. SIAM J. Sci. Comput 21(6), 19691972 (2000)
[51]Paige, C.C., Saunders, M.A.: Solutions of sparse indefinite systems of linear equations. SIAM J. Numer. Anal 12(4), 617629 (1975)
[52]Pearson, J.W., Stoll, M., Wathen, A.: Preconditioners for state constrained optimal control problems with Moreau-Yosida penalty function. Numerical Linear Algebra with Applications 21(1), 8197, (2014)
[53]Pearson, J.W., Stoll, M., Wathen, A.J.: Regularization-robust preconditioners for time-dependent PDE-constrained optimization problems. SIAM J. Matrix Anal. Appl 33, 11261152 (2012)
[54]Pearson, J.W., Wathen, A.J.: A new approximation of the Schur complement in preconditioners for PDE-constrained optimization. Numerical Linear Algebra with Applications 19, 816829 (2012). DOI 10.1002/nla.814. URL
[55]Peirce, A., Dahleh, M., Rabitz, H.: Optimal control of quantum-mechanical systems: Existence, numerical approximation, and applications. Physical Review A 37(12), 4950 (1988)
[56]Pironneau, O.: Optimal shape design for elliptic systems. System Modeling and Optimization pp. 4266 (1982)
[57]Rees, T., Dollar, H.S., Wathen, A.J.: Optimal solvers for PDE-constrained optimization. SIAM Journal on Scientific Computing 32(1), 271298 (2010). DOI
[58]Rees, T., Stoll, M., Wathen, A.: All-at-once preconditioners for PDE-constrained optimization. Kybernetika 46, 341360 (2010)
[59]Reyes, De los, Juan-Carlos, and Carola-Bibiane, Schönlieb: Image denoising: learning noise distribution via PDE-constrained optimization, (2012)
[60]Ruge, J. W. and Stüben, K.: Algebraic multigrid. in Multigrid methods, vol. 3 of Frontiers Appl. Math., SIAM, Philadelphia, PA, 1987, pp. 73130.
[61]Saad, Y.: Iterative methods for sparse linear systems. Society for Industrial and Applied Mathematics, Philadelphia, PA (2003)
[62]Saad, Y.: A flexible inner-outer preconditioned GMRES algorithm. SIAM Journal on Scientific Computing, 14 (1993), pp. 461461.
[63]Saad, Y., Schultz, M.H.: GMRES: A generalized minimal residual algorithm for solving non-symmetric linear systems. SIAM J. Sci. Stat. Comput., 7(3), 856869 (1986).
[64]Simoncini, V., Szyld, D.: Recent computational developments in Krylov subspace methods for linear systems. Numer. Linear Algebra Appl 14(1), 161 (2007).
[65]Stoll, M.: All-at-once solution of a time-dependent time-periodic PDE-constrained optimization problems. IMA J Numer Anal (2013)
[66]Stoll, M., Wathen, A.: All-at-once solution of time-dependent PDE-constrained optimization problems. Technical Report, University of Oxford, (2010)
[67]Strang, G., Fix, G.: An Analysis of the Finite Element Method 2nd Edition, 2nd edn. Wellesley-Cambridge (2008)
[68]Takacs, S., Zulehner, W.: Convergence analysis of multigrid methods with collective point smoothers for optimal control problems. Computing and Visualization in Science 14, 131141 (2011)
[69]Tröltzsch, F.: Optimal Control of Partial Differential Equations: Theory, Methods and Applications. Amer Mathematical Society (2010)
[70]Ulbrich, M.: Semismooth Newton Methods for Variational Inequalities and Constrained Optimization Problems. SIAM Philadelphia (2011)
[71]Van Der Vorst, H.A.: BI-CGSTAB: A fast and smoothly converging variant of BI-CG for the solution of nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 13(2), 631634 (1992).
[72]Wathen, A.J., Rees, T.: Chebyshev semi-iteration in preconditioning for problems including the mass matrix. Electronic Transactions in Numerical Analysis 34, 125135 (2008)
[73]Wachsmuth, D. and Wachsmuth, G.: Necessary conditions for convergence rates of regularizations of optimal control problems, RICAM Report 4 (2012)
[74]Wathen, A.J.: Preconditioning and convergence in the right norm. International Journal of Computer Mathematics, 84 (2007), pp. 11991209.
[75]Wesseling, P.: An introduction to multigrid methods. Pure and Applied Mathematics (New York), John Wiley & Sons Ltd., Chichester, 1992.
[76]Wilson, J. and Patwari, N. and Vasquez, F.G.: Regularization methods for radio tomographic imaging. 2009 Virginia Tech Symposium on Wireless Personal Communications (2009)
[77]Zulehner, W.: Analysis of iterative methods for saddle point problems: a unified approach. Math. Comp 71(238), 479505 (2002)
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Communications in Computational Physics
  • ISSN: 1815-2406
  • EISSN: 1991-7120
  • URL: /core/journals/communications-in-computational-physics
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


MSC classification


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed