A computer-based method controls a dynamical system in an uncertain environment within a bounded probability of failure. The dynamical system has a state space and a control space. The method includes diffusing a risk constraint corresponding to the bounded probability of failure into a martingale that represents a level of risk tolerance associated with the dynamical system over time. The state space and the control space of the dynamical system are augmented with the martingale to create an augmented model with an augmented state space and an augmented control space. The method may include iteratively constructing one or more Markov Decision Processes (MDPs), with each iterative MDP represents an incrementally refined model of the dynamical system. The method further includes computing a first solution based on the augmented model or, if additional time was available, based on one of the MDP iterations.
