Abstract. We develop algorithms for the computation of the distribution of the total reward accrued during [0,t) in a finite continuous-parameter Markov chain. During sojourns, the reward grows linearly at a rate depending on the state visited. At transitions, there can be instantaneous rewards whose values depend on the states involved in the transition. For moderate values of t, the reward distribution is obtained by implementing a series representation, due to Sericola, that is based on the uniformization method. As an alternative, that distribution can also be computed by the numerical inversion of its Laplace-Stieltjes transform. For larger values of t, we implement a matrix convolution product to compute a related semi-Markov matrix efficiently and accurately.

  <postscript>  file of the paper

Back to Beatrice Meini's home page.