AI Computing and Large-Scale AI Foundation Model

AI 運算 與 大規模 AI 基礎模型

For AI training, due to the “emergent” abilities of large-scale models, the parameters of neural networks have grown from hundreds of millions (~100Mn) to hundreds of billions (~100Bn) , such as OpenAI GPT-3 (175Bn). Moreover, some researchers started to train brain-scale AI models with tens of trillions parameters. Huge training computing demands lead to exa-scale AI supercomputer. Those AI applications and computing requirements bring tough technology challenges, but also open new opportunities for domain-specific computing architecture for IC design.