This thesis explores advanced deep reinforcement learning methods for learning dialog policies. While many recent contributions in the area of reinforcement learning focus on learning how to play Atari games, this thesis applies them in a real-world scenario. When talking to a dialog system, the dialog policy is the component which chooses the response based on the history of the interaction between user and system. Nowadays, dialog policies may be learned automatically by training a reinforcement learning agent with a user simulator. In this thesis, a baseline method for dialog policy learning is implemented and extended by various state-of-the art deep reinforcement learning methods. An ablation study discusses the significance of each extension, highlighting beneficial and harmful additions. Each extended agent is shown to perform better than the baseline method with all agents outperforming policies from an existing benchmark. Two agents even prove to be on par with handcrafted dialog policies. Along with the quantitative evaluation, qualitative results are provided in the form of chats between a user and a trained agent.