Markov chain, Random walk, Markov Decision Processes
Abstract
The research paper discusses the significance of Markov chain and random walk as fundamental probabilistic models that describe transitions between states in various fields, including natural language processing, finance, and bioinformatics. It traces the historical development of random walk, beginning with Karl Pearson’s initial concept in 1905 and culminating in the emergence of quantum random walks in the 21st century, while highlighting their mathematical properties and applications. The paper categorizes states in Markov chain into recurrent and transient, providing criteria for their classification and demonstrating the recurrence of random walks in different dimensions. It explores diverse applications of random walk, such as knowledge representation learning, Markov Decision Processes in reinforcement learning, advancements in medical research through neuroimaging, and innovative strategies for waste recycling path planning. The conclusion emphasizes the ongoing evolution of random walk theories, particularly with advancements in quantum computing and big data, suggesting that their applications will continue to expand and become increasingly sophisticated in the future.
