Discrete Stochastic Processes

Focusing on discrete-time stochastic processes, the text explores random interactions and algorithms centered on the Markov property. It delves into topics such as random walks, Markov chain convergence, and phase-type distributions, with practical applications in search engines and probabilistic automata. The introduction of the Ising model highlights its relevance in statistical physics. Additionally, it addresses data science applications through hidden Markov models and decision processes. The book includes 32 exercises and 17 detailed problems, enhancing understanding of statistical learning concepts.