What is a Markov Chain?

Question

usaho4w · Accepted Answer

A stochastic process which is memoryless.
Explanation:
Suppose you have a system that changes state over time, with some random variability. The time sequence of states of the system is called a stochastic process. A stochastic process is a Markov chain if at any point in time, the probability of future states is only dependent on the current state, not on anything that has gone before. Another term used is memoryless. The system does not remember what happened before in the sense that future states are independent of past states.
Consider a model of Brownian motion:
There are n particles of pollen which we keep track of.
The particles of pollen tend to keep moving in the same direction with the same velocity.
The movement is changed by the random impact of unseen smaller particles which we are not tracking.
If our model only maintained a state which only described the instantaneous position of each particle of pollen at each time t then this would not be a Markov chain: The future position of each particle is influenced by the current velocity of the particle too - and probably its angular momentum.
If however the model maintained a state which described not only the position of each pollen particle, but also its current velocity and angular momentum then it probably is a Markov chain.

Edward Patten · Accepted Answer

Markov chain is a state of any particular situation that is possible in the system. It refers to any system where there are a certain number of given probabilities and states that the system can change from any state to another state. As for example, if you are studying rainy days, then two states are there: It is not raining today, and it is raining today. A simple weather model can be illustrated by the transition matrix.

P = [\begin{matrix} 0.9 & 0.1 \\ 0.5 & 0.5 \end{matrix}]

The P matrix is the weather model in which a rainy day is 50% likely to be followed by another rainy daya, and a sunny day is 90% likely to be followed by another sunny day.

What is a Markov Chain?

Answered question

Answer & Explanation

New Questions in College Statistics