Simulation of visual impairments may be used to counter design exclusion and improve the accessibility of products. Whilst first-hand experience of the simulated scene is important, having to manually look for differences caused by vision deficiencies, let alone judge their severity, is time-consuming and error-prone. In this work, we develop an error metric that tracks errors and uncertainty throughout the stages of a simulator for visual impairments. This allows us to visualize errors and uncertainty independently in all tracked dimensions. We add more advanced view modes to the simulator to enable visualization of our error metric in combination with the input and output images. We further extend the simulator to include simulations for strabismus, astigmatism and retinal ganglion cells. By simulating several combinations of vision deficiency, we found that a visualization of the proposed metric can be used to identify problematic areas in a scene. Depending on the use case, one may need to select different combination functions generating scalar values, e.g. the aggregated standard deviation of the RGBXY vector. With a comparison of our metric to SSIM, we found that our metric can cope better with displaced features of an image but may produce blurred visualizations. We conclude that although our metric imposes considerable performance penalties on the simulator, it has advantages compared to approaches based exclusively on input and output images.