AI: Chips and talent are lacking in the Chinese attempt – 05/10/2023 – Market

As China’s AI industry develops, companies face a looming challenge: a shortage of chips.

Yin Qi, co-founder and CEO of Megvii Technology Ltd., one of China’s leading artificial intelligence (AI) companies, told Caixin news agency in an interview at the end of March that there are only about 40,000 graphics processing units (GPUs) in China. ) Datacenter-grade Nvidia A100, the kind used to build large-scale machine learning infrastructure.

Companies connect computers equipped with these chips to create powerful clusters (groups) of GPUs to run advanced programs, including those related to AI and machine learning – with more chips meaning more processing power.

For a company to build a large GPT language model, at least 10,000 A100 chips would be needed, Yin said. And due to chip shortages, companies in China would only be able to create GPU clusters with around 3,000 to 5,000 chips, he added. For an AI-related comparison, it would take more than 30,000 A100 GPUs to run OpenAI’s ChatGPT, according to market researcher TrendForce.

Not to be outdone, companies building great AI models have increased their budgets for these industry-leading Nvidia GPUs. In China, demand for these chips has been driven by companies that train generative AIs like ChatGPT.

However, Chinese companies have not been able to get their hands on these chips because last year the US government prevented Nvidia from exporting the A100 and its newest data center chip, the H100 – which are essential for the development of large models. language and generative AI – for Chinese customers.

Chinese companies can only buy the A800 chip, a scaled-down version of the A100 whose chip-to-chip data transfer rate is two-thirds of the latter, which can restrict the overall computing power of any application running on the chip.

However, that bone thrown by Nvidia won’t alleviate China’s chip shortage on its own. An executive at a chip design company listed in mainland China predicted that it would be a challenge to meet demand from all Chinese companies, given the explosive pace of development.

There are domestic alternatives to GPUs from Nvidia and its main competitors, Advanced Micro Devices and Intel. Chinese GPU startup Shanghai Biren Intelligent Technology, considered a promising opponent for Nvidia, has impressive computing power but lacks the speed at which data can be transferred between two devices or systems.

Still, even if Chinese companies could design chips capable of competing with or surpassing the A100, they wouldn’t be able to find a foundry to manufacture them. In October, the US Department of Commerce issued new regulations that would restrict wafer foundries’ access to US technology if they produce advanced chips for Chinese customers.

Furthermore, over the years, mainland GPU manufacturers have come to rely heavily on Taiwanese companies for manufacturing and packaging processes. They also rely on US companies for electronic design automation tools – key software that allows developers to design, model, simulate and test circuit designs before production.

Of the three main elements of AI development – ​​data, algorithms and computing power – the US is targeting computing power to contain the advance of the Chinese AI industry. Developers won’t be able to tweak algorithms without enough computing power, which would render large amounts of data useless, said a person familiar with US semiconductor export control policy.

While Chinese companies have a wealth of data at their disposal, there is a consensus that it lacks quality. A US investor noted that Chinese companies have to spend a significant amount of time and manpower to clean up the datasets needed for machine learning. So, to save time, many large Chinese models are trained on US datasets, which can leave Chinese companies with problematic AI to generate answers that require knowledge about local norms and contexts.

For example, with Baidu’s Ernie Bot imaging feature, some people complained that the command was translated into English before a response was generated, and therefore failed to understand things like Chinese idioms and names of certain foods. locations.

China faces another hurdle in the global AI battleground: a lack of top talent. While Chinese universities have opened more AI courses, the Chinese supply of AI experts is not meeting the demand.

The country currently lacks about 300,000 AI professionals, according to a report by the Chinese Academy of Labor and Social Security. Areas experiencing the greatest shortages include AI chip design, machine learning, natural language processing, algorithm research, and application development.

In terms of research talent, China held 232 spots on the 2022 list of the world’s most influential AI academics, ranked by Tsinghua University and other institutions. By comparison, the US pollsters occupied 1,146 spots on the list of nearly 1,900 people.

The United States also produced the most technical talent for the AI ​​industry: 39.4% of the world’s AI professionals in 2020, followed by India and the United Kingdom. China ranked fourth, with less than 5% of the total, according to a report by Jean-François Gagné, an AI entrepreneur and founder of Element. AI. Technical roles include research, data engineering, AI engineering and production, and machine learning.

China knows it needs to produce more AI experts if it wants to stand a chance. In its report, the Chinese Academy of Labor and Social Security urged the country to better align its AI talent cultivation efforts with industry needs, draw on the experience of the world’s top universities, and establish innovative teaching models to increase its reserve. of AI talent.

In addition to bolstering their pipeline of talent, Chinese companies, governments and universities can also consider how to improve their institutional mechanisms and find innovative ways to improve the work environment to attract and retain AI talent.

Translated by Luiz Roberto M. Gonçalves