Repository: Freie Universit├Ąt Berlin, Math Department

A sparse Markov chain approximation of LQ-type stochastic control problems

Banisch, Ralf and Hartmann, C. (2015) A sparse Markov chain approximation of LQ-type stochastic control problems. Math. Control Relat. F. . ISSN 1064-8275 (In Press)

[img]
Preview
PDF - Submitted Version
1932Kb

Abstract

We propose a novel Galerkin discretization scheme for stochastic optimal control problems on an indefinite time horizon. The control problems are linear-quadratic in the controls, but possibly nonlinear in the state variables, and the discretization is based on the fact that problems of this kind admit a dual formulation in terms of linear boundary value problems. We show that the discretized linear problem is dual to a Markov decision problem, prove an $L^{2}$ error bound for the general scheme and discuss the sparse discretization using a basis of so-called committor functions as a special case; the latter is particularly suited when the dynamics are metastable, e.g., when controlling biomolecular systems. We illustrate the method with several numerical examples, one being the optimal control of Alanine dipeptide to its helical conformation.

Item Type:Article
Subjects:Mathematical and Computer Sciences > Mathematics > Numerical Analysis
Divisions:Department of Mathematics and Computer Science > Institute of Mathematics
Department of Mathematics and Computer Science > Institute of Mathematics > Cellular Mechanics Group
Department of Mathematics and Computer Science > Institute of Mathematics > BioComputing Group
ID Code:1507
Deposited By:Carsten Hartmann
Deposited On:18 Feb 2015 00:30
Last Modified:25 Nov 2015 10:56

Repository Staff Only: item control page


  • Mathematics