Deep learning methods have successfully advanced many fields of research with their ability to learn complex features from data. While they have been used successfully in BCI research, their use for cognitive science, where the increased complexity of deep learning methods could reveal novel insights about how our brain functions, is just starting to be explored. In this thesis, we look at three established EEG decoding models, EEGNet, Shallow ConvNet, and Deep ConvNet, on the ERP CORE dataset, which includes seven different ERP components. We will look at how parameters like model architecture, sample count, and preprocessing affect decoding accuracies, compare subject accuracies across the different ERP paradigms, and look at how feature attribution can be used to explain the decisions of our networks, as well as gain new insights into cognitive processes. We conclude that that deep learning can be a valuable tool for cognitive science that needs further research to reach its full potential.