A recent study [WGW19] has provided clues that conscientiousness can have a negative effect on the performance in solving coding challenges. Since coding challenges have become a popular tool to assess the problem solving ability and conscientiousness is widely acknowledged to be a positive influence on work performance this has serious implications. To study this effect and its consequences we conducted an exploratory study to find differences less and more conscientious developers display while solving coding challenges. Further, we analyze the differences found on their impact on the performance in solving coding challenges. Our findings indicate that software developers of intermediate and high conscientiousness are more likely to create concepts, think in silence for longer periods of time, start implementing later than less conscientious software developers and provide better code quality. Furthermore, we found that software developers use trial and error approaches regardless of their level of conscientiousness.