As the vision of fully autonomous vehicles potentially introduces significant benefits for our society, this work investigates approaches for sequential decision making for high-level actions in highway scenarios. These scenarios are modeled using an markov decision process (MDP) and consider deep reinforcement learning to solve it. Our approach, based on deep Q-networks (DQNs), is able to fully avoid collisions and learns a policy that results in comfortable trajectories compared to baseline policies we developed. One of the main challenges for reinforcement learning are sparse rewards, which we aim to overcome employing reward shaping. Additionally, the necessity of multiple layers of non-liniearities in the DQN algorithm is empirically evaluated using our scenarios. The results support the usage of multiple levels of non-linearities, as a linear variant of the DQN is not capable of learning effective policies in our experiments. Due to a weight initialization with behavioral cloning, an acceleration of the learning procedure is achieved.