This thesis investigates human-robot interactions focusing on theoretical reasoning about a general formalism that is suitable to describe most HRI systems. We belief that the question whether to ask the human for help or not is a decision that should be made at the meta level. This work summarizes the milestones in the literature that contribute to HRI, compares different MDP concepts to choose from and proposes a new formalism along with several analysis' on its properties.