The fundamental idea of Reinforcement Learning (RL) is learning by interacting with an environment through trial and error and is inspired by the way animals learn in nature. However, standard RL approaches still struggle with seemingly simple tasks, where humans would excel after a few trials. In detail, there is a high volatility between different runs of the RL training process, in addition to a slow convergence rate for each trial. Previous work has shown that using a biologically inspired approach to RL significantly improves this learning speed and increases the robustness of training across different runs. The biologically based approach uses a state representation, called Spatial Semantic Pointers (SSPs), where embeddings of the state are encoded in a biologically plausible vector space representation. Experiments on a simple two-dimensional navigation task have shown that introducing a grid-like structure into the vector space further increases the learning speed. However, it remains unclear, whether these findings scale to different environments with more complex inputs. Specifically, we are interested in comparing this approach in environments with state spaces to environments with more complex inputs, such as RGB images. Furthermore, the approach is also compared to common artificial neural networks with the state-of-the-art Advantage Actor-Critic (A2C) agent. Our results suggest that using biologically based state representations leads to a faster learning speed in some environments while causing slower learning in others. Further, the state representations do not extend very well on larger or more complex inputs like images, causing worse performance in both learning speed and overall training time needed.