# Chapter 3: Finite Markov Decision Processes

* You take a action a\_t at a time-step in state s\_t and in response to that action you get response(reward)  r\_t+1 just after that, in next time step and also get into new state s\_t+1.
* S0, A0, R1, S1, A1, R2, S2, A2, R3, . . .  this sequence defines a MDP.&#x20;
* Markov Property:- Future is independent of the past given the present.
* Finite MDP - Have finite element in S,A,R set
*

<div align="left"><img src="https://1877261540-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LFDuA0A2VRqmT31Blrq%2F-LFfR6UHh8MQ2aDuvqhl%2F-LFf_IwiGUrj4NDy5Of7%2Fimage.png?alt=media&#x26;token=ff94aecc-1694-4ac6-a9e7-e5d286c19877" alt="p defines the dynamics of the MDP"></div>

<div align="left"><img src="https://1877261540-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LFDuA0A2VRqmT31Blrq%2F-LFfR6UHh8MQ2aDuvqhl%2F-LFfbvtXoohNrvrzOhLw%2Fimage.png?alt=media&#x26;token=e00ce679-cd8e-4c7c-85b8-588dabed3f89" alt="State transition probabilities"></div>

<div align="left"><img src="https://1877261540-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LFDuA0A2VRqmT31Blrq%2F-LFfR6UHh8MQ2aDuvqhl%2F-LFfc8GGY9BNY9NmVr_5%2Fimage.png?alt=media&#x26;token=bfa7e420-b0a9-4793-a707-bbc5f67b9324" alt="reward"></div>

![bellman recursive equation](https://1877261540-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LFDuA0A2VRqmT31Blrq%2F-LFfjs5PDWqsnGvI3_s0%2F-LFfjteNOF-Elznx8pc-%2Fimage.png?alt=media\&token=090b9b4c-bdba-4da8-8097-eed529436f32)

