LLMs like ChatGPT can perform data analysis at a fraction of the human cost with comparable performance: study
- Preliminary findings are in study of LLMs by Alibaba’s Damo Academy and Singapore’s Nanyang Technological University
- The cost of GPT-4, the latest version of the LLM developed by US start-up OpenAI is only 0.45 per cent of hiring a senior data analyst, study finds
Using large language models (LLM) like GPT-4 in data analysis costs less than 1 per cent of hiring a human analyst while turning in comparable performances, according to a recent study that highlights the potential threat to job security amid increased adoption of generative artificial intelligence (AI) in various industries.
Alibaba owns the South China Morning Post.
In the study, the researchers gave questions and provided the data, and automated the whole data analytics process using GPT-4 to extract and analyse data, to eventually produce insights and graphs.
The results of the analysis were compared with professional human data analysts in terms of performance, time and costs.
04:48
Will ChatGPT replace reporters? We asked AI to write for the Post
The experiments showed that GPT-4 is not just significantly cheaper than a human data analyst, but also much faster in completing the tasks, according to the study findings.
However, the study noted that further studies are needed before concluding that GPT-4 can replace data analysts.
GPT-4 can also beat an entry-level human analyst in terms of performance, which was evaluated through a range of metrics including the correctness and fluency in charts and the insights they produced, and has “comparable performance” to a senior level analyst, with advantages varying among different cases and metrics.
In some cases, the AI model managed to surpass the human data analysts in terms of the correctness of the figures and analysis, and the insights GPT-4 generated tended to be more complex, according to the study.
GPT-4 also got full marks in alignment and fluency in generating the analysis with grammatically correct texts.
However, GPT-4 fell behind humans in terms of showing correct data in graphs, as well as presentation and formatting in some cases.
Despite errors with some figures, GPT-4 could still generate correct analysis, the study pointed out.
The study sheds new light on the expanding application of LLMs in various industries, where it is expected to improve efficiency but also threaten human jobs.
Chinese tech companies are racing to develop their own ChatGPT-like services.
