US researchers have developed an open-source dataset designed to improve how AI models interpret charts, with potential use in analysing scientific figures and business trends.
The Massachusetts Institute of Technology (MIT) and the MIT-IBM Watson AI Lab team said ChartNet contains 1.5 million chart samples spanning 24 chart types and six plotting libraries. The paper, ‘ChartNet: a million-scale, high-quality multimodal dataset for robust chart understanding’, was presented at June’s IEEE Computer Vision and Pattern Recognition Conference.
The researchers said existing vision-language models can struggle with charts because they must combine visual, numerical and linguistic understanding. Models trained with ChartNet improved across chart reconstruction, data extraction, summarisation and chart question-answering, the team said.
Lead author Jovana Kondic, an MIT electrical engineering and computer science graduate student, said the aim was to help smaller models achieve strong performance without requiring “infinite amounts of computation”.
