Nowadays, we use various types of charts in academic papers, journals, and newspapers for several purposes. Information Visualizations are powerful tools that researchers use to describe the data. It makes it easier to detect hidden information and patterns from the data, such as trends and relationships. However, the huge increase in data amount and complexity leads to escalating the number of visualizations as well and complicating its design process. Designers should analyze all those factors to choose the best design decisions for readable and straightforward visualization. For this reason, researchers have been using machine learning approaches in building automated systems that extract information and attributes from visualizations and infographics. The ML-based automated system would simplify the evaluation of visualizations which makes it easy to automate generating charts from data, based on the visual encodings it has learned. This project explores the research problem that states the ability of the machine to form a good representation of heterogeneous charts. These representations let the users compare and classify them from a meaningful human perspective. We aim to apply unsupervised machine learning methods on charts image to get a simplified representation from heterogeneous charts. Our method applies transfer knowledge methods in computer vision. It uses a pre-trained CNN on the ImageNet dataset to get a chart representation vector and uses dimension reduction methods on the network output to project all charts representation on a 2D plane. We evaluate this approach by applying it on different use-case scenarios of different charts’ datasets that explain the projection results and determines the context of the distance measure on the output space.