Scientists at the Korea Advanced Institute of Science and Technology (KAIST) have unveiled an AI chip that they claim matches the speed of Nvidia's A100 GPU, is smaller in size, and consumes significantly less power. This chip was developed using Samsung's 28 nanometer manufacturing process. This technology is considered relatively old in the rapidly changing world of semiconductors.
The team, led by Professor Yu Hui-jun from KAIST's In-Memory Processing Research Center, has developed what is said to be the world's first “complementary transformer” (C-trans) AI chip. This neuromorphic computing system uses deep learning models commonly used in visual data processing to mimic the structure and workings of the human brain.
“Neuromorphic computing is a technology that even companies like IBM and Intel have not been able to implement, and we are proud to be the first in the world to run LLM on a low-power neuromorphic accelerator.” Yoo said.
doubts remain
This technology learns context and meaning by tracking relationships in data, such as words in sentences. This is a key technology for generative AI services like ChatGPT.
During a demonstration at the Ministry of ICT's headquarters, team member Kim Sang-yeop demonstrated the chip's capabilities. He performed tasks such as his LLM in OpenAI, his Q&A sessions using GPT-2, text summarization, and translations on a laptop equipped with the chip. The task completed at least three times faster for him than when he runs GPT-2 on a laptop connected to the Internet, and in some cases up to nine times faster.
Implementing LLM for generative AI tasks typically requires a large number of GPUs and 250 watts of power, but the team claims that the semiconductor uses only 1/625th the power of Nvidia's GPUs for the same task. doing. Additionally, the size is 4.5mm x 4.5mm, which is 1/41, so it may eventually be applied to mobile phones and other devices.
However, it remains to be seen whether this chip can deliver on its promise in real-world applications.as tom's hardware “The KAIST C-Transformer chip is said to be able to perform the same LLM processing tasks as one of Nvidia's powerful A100 GPUs, but no direct comparative performance metrics were provided in press or conference materials. This is an important statistic, conspicuous by its absence, and a cynic would probably surmise that the performance comparison doesn't do the C transformer any favors.”
