Markov

Markov Property

The likelihood of a future state depends on present state only.

$$P(X_t|X_{t-1}, X_{t-2},...,X_1) = P(X_t|X_{t-1})$$

Markov Chain

A stochastic process with Markov property.

Hidden Markov Model (HMM)

The state is not directly visible, but output dependent on the state is visible.