Banisch, Ralf and Hartmann, C. (2015) *A sparse Markov chain approximation of LQ-type stochastic control problems.* Math. Control Relat. F. . ISSN 1064-8275 (In Press)

| PDF - Submitted Version 1932Kb |

## Abstract

We propose a novel Galerkin discretization scheme for stochastic optimal control problems on an indefinite time horizon. The control problems are linear-quadratic in the controls, but possibly nonlinear in the state variables, and the discretization is based on the fact that problems of this kind admit a dual formulation in terms of linear boundary value problems. We show that the discretized linear problem is dual to a Markov decision problem, prove an $L^{2}$ error bound for the general scheme and discuss the sparse discretization using a basis of so-called committor functions as a special case; the latter is particularly suited when the dynamics are metastable, e.g., when controlling biomolecular systems. We illustrate the method with several numerical examples, one being the optimal control of Alanine dipeptide to its helical conformation.

Item Type: | Article |
---|---|

Subjects: | Mathematical and Computer Sciences > Mathematics > Numerical Analysis |

Divisions: | Department of Mathematics and Computer Science > Institute of Mathematics Department of Mathematics and Computer Science > Institute of Mathematics > Cellular Mechanics Group Department of Mathematics and Computer Science > Institute of Mathematics > BioComputing Group |

ID Code: | 1507 |

Deposited By: | Carsten Hartmann |

Deposited On: | 18 Feb 2015 00:30 |

Last Modified: | 25 Nov 2015 10:56 |

Repository Staff Only: item control page