GERSENDE FORT, SEAN MEYN, ERIC MOULINES, AND PIERRE PRIOURET

Abstract:

Fluid limit techniques have become a central tool to analyze queueing net- works over the last decade, with applications to performance analysis, simulation, and optimization. In this paper some of these techniques are extended to a general class of skip-free Markov chains. As in the case of queueing models, a fluid approximation is obtained by scaling time, space, and the initial condition by a large constant. The resulting fluid limit is the solution of an ordinary differential equation (ODE) in “most” of the state space. Stability and finer ergodic properties for the stochastic model then follow from stability of the set of fluid limits. Moreover, similar to the queueing context where fluid models are routinely used to design control policies, the structure of the limiting ODE in this general setting provides an understanding of the dynamics of the Markov chain. These results are illustrated through application to Markov Chain Monte Carlo.

Reference:

@inproceedings{formoumeypri06,
author = {G. Fort and E. Moulines and S. Meyn and P. Priouret},
title = {ODE methods for Markov chain stability with applications to MCMC},
booktitle = {Valuetools '06: Proceedings of the 1st international conference on Performance evaluation methodolgies and tools},
year = {2006},
isbn = {1-59593-504-5},
pages = {42},
location = {Pisa, Italy},
doi = {http://doi.acm.org/10.1145/1190095.1190149},
publisher = {ACM Press},
address = {New York, NY, USA},
note={Full version to appear, Annals of Applied Probability}}