A Markov decision process (MDP) is a 4-tuple (a set of states, a set of actions, a set of rewards, and a transition probability function). MDPS are used to study a wide range of randomization problems. MDPs find optimal solutions to sequential and stochastic decision problems. An MDP binds previous, current, and future system decisions through the proper definition of system states. The use of MDPs for modeling and solving medical treatment decisions has been increased significantly in recent years. This study gives an overview of MDP models and solution techniques.We describe MDP modeling in the context of healthcare data.