China unleashes might of AI on next-generation supercomputer to meet demand for more power
- Supercomputer recorded a 75,839-fold boost in handling data for machine learning after a breakthrough in memory management
- China’s decision to keep quiet about its computing advances has prompted speculation overseas
Aided by a breakthrough in memory management technology, the New Generation Sunway supercomputer developed by the National Research Centre of Parallel Computer Engineering and Technology recorded a 75,839-fold boost in handling data for machine learning. The overall performance of the computer increased 88 times when processing some of the most challenging AI-related tasks.
“The increasing demand for computing power has prompted the rapid development of supercomputers in recent years,” said Zhai Jidong, associate professor of computer science and his colleagues with Tsinghua University in a paper published in the peer-reviewed Journal of Tsinghua University on Wednesday.
“Traditional supercomputers mainly serve scientific computing applications. Their support for machine learning, graph computing and big data processing is poor, limiting the application scenarios.”
Using the new technology, the Sunway exascale machine “is running many AI applications, including large-scale machine learning and molecular dynamics simulation”, they added.
An exascale computer can be 1,000 times more powerful than existing mainstream supercomputers. However, these machines are not just about speed.
In the past, a supercomputer’s job was calculation. The performance limit was largely set when the computer was built.
With the increasing use of artificial intelligence technology in many areas, the potential users of exascale computers hope these giant, expensive machines will also run AI algorithms that learn from their jobs and get smarter over time.
But Zhai and colleagues reported that when they ran an AI tool on Sunway that could be used by the military to identify targets in satellite photos, the powerful machine struggled with an efficiency poorer than a desktop computer.
The problem was caused by a bottleneck in the computer’s memory management process, according to their study.
Sunway was built to handle traditional calculation tasks, such as modelling a nuclear explosion. Before doing the calculation, the computer would receive a certain amount of data, break it into chunks and allocate them properly to its memory slots.
An AI algorithm operates with more uncertainties. It is constantly fed new data sets, often of vastly different sizes, that can quickly disrupt the supercomputer’s memory.
Swalloc, the new memory allocator developed by the Tsinghua team, worked like an alchemist’s pot. It could transform different data sets into a uniform format and allocate them efficiently to the supercomputer’s memory slots for fast, easy access by AI algorithms.
Other teams in China have achieved similar breakthroughs. In July, Tianhe 3, an exascale computer developed by the National University of Defence Technology in Tianjin won two top international prizes – SSSP Graph500 and Big Data Green Graph500 – for AI application in supercomputers.
The Graph500 steering committee comprises more than 50 international high-performance computing experts from academia, industry and national laboratories.
Tianhe 3 used a home-developed processor with hardware acceleration for an AI algorithm that outperformed A100 – the fastest Nvidia GPU widely used in the construction of exascale computers in Western countries – according to the Chinese military researchers.
China has built more supercomputers than any other country, and twice the number of those in the United States. They played an important role in the rapid development of nuclear warheads, hypersonic weapons, large-scale infrastructure, life science and global monitoring networks.
The New Generation Sunway and Tianhe 3 reached exascale performance – one quintillion calculations per second – in March and became the first machines with such capabilities in the world, according to The Next Platform, a leading website reporting on high-performance computing.
Chinese authorities did not submit the results to Top500, a list of the world’s most powerful computers. China’s decision to keep quiet about its achievement has prompted speculation overseas about the impact of US sanctions.