Chinese AI chips fall short on coding, forcing firms to stretch scarce Nvidia supply
Chinese AI companies are struggling to use domestic chips for complex tasks like coding, forcing them to rely on a limited supply of high-end Nvidia processors for critical inference workloads.
Intelligence analysis by Gemini 2.5 Flash
As artificial intelligence moves into large-scale deployment, Chinese firms face a significant compute constraint. While domestic chips can handle some inference tasks, high-tier applications demanding stringent performance, such as coding, still necessitate Nvidia's advanced hardware, leading to a 'bipolarisation' of demand and challenges in commercial viability for domestic solutions.
Imagine you have a super-smart robot brain that needs to do two things: first, learn everything (that's 'training'), and second, use what it learned to answer questions or do jobs (that's 'inference'). China has some good robot brains for learning, but when it comes to the really tricky jobs, like writing computer code, their own brains aren't quite smart enough. So, they still need to borrow special, super-smart brains from a company called Nvidia, but there aren't many to go around. This means they can do easy jobs with their own brains, but the hard, important ones still need the special Nvidia brains.
Analysis
The current state of China's AI chip landscape reveals a critical dependency on foreign technology, particularly Nvidia's high-end processors, for advanced AI inference tasks. While the nation has made strides in developing its own hardware, these domestic solutions are primarily relegated to lower-tier inference workloads, which offer limited commercial viability and monetization opportunities. This creates a significant bottleneck for Chinese AI companies aiming to deploy sophisticated models that require high computational precision and performance.
Guan Jiawei
Guan Jiawei, vice-president of inference optimisation start-up Approaching.AI, articulates the core challenge as a "bipolarisation" of demand within the Chinese AI sector. He notes a stark contrast between the surging demand for high-quality tokens—the fundamental units of data processed by AI models—and the available supply that can meet stringent performance metrics. According to Guan, domestic processors are currently unable to reliably deliver the performance required for these high-tier tasks, especially in critical scenarios like coding, where users are willing to pay a premium for accuracy and efficiency. This assessment underscores the qualitative gap that Chinese chips still need to bridge.
Guan further emphasizes the commercial implications of this disparity, stating that relying solely on domestic chips for inference limits firms to handling "low-quality tier" tasks. These tasks are characterized by weak demand and poor monetization potential, making it exceedingly difficult for companies to establish a sustainable commercial path. His insights highlight that the economic viability of advanced AI applications in China remains intrinsically linked to the availability and performance of Nvidia's hardware, which continues to be the benchmark for high-quality token processing.
Approaching.AI
Approaching.AI, as an inference optimisation start-up, represents a segment of the Chinese tech industry actively seeking to mitigate the compute squeeze through software innovation. Their efforts are focused on optimising software to cope with the surging demand for inference, particularly as AI models transition from development to large-scale deployment. This approach aims to maximize the efficiency of existing hardware resources, including the limited pool of high-end chips and potentially improving the utility of domestic processors for certain tasks.
The company's work is crucial in a landscape where access to advanced foreign chips is restricted. By enhancing software capabilities, firms like Approaching.AI can help stretch the utility of scarce Nvidia supplies and potentially enable domestic hardware to handle a broader range of inference tasks, even if not the most complex ones. This strategic focus on software optimization is a pragmatic response to hardware limitations, seeking to extract more performance from available resources and reduce the immediate reliance on external chip imports.
140 trillion
The sheer scale of AI activity in China is underscored by the National Data Administration's report that the country's average daily token calls exceeded 140 trillion in March. This figure represents an astonishing more than 1,000-fold increase from the beginning of 2024, illustrating the explosive growth in AI usage and the corresponding demand for computational power. This skyrocketing token usage is exacerbated by AI models becoming more "agentic," meaning they are performing real-world tasks rather than merely answering questions, which inherently requires more intensive and precise processing.
This exponential increase in token usage directly translates into an acute compute squeeze, intensifying the pressure on Chinese AI firms to secure adequate processing capabilities. The demand for high-quality tokens, essential for these agentic AI applications, far outstrips the supply that domestic chips can reliably provide. The figure of 140 trillion daily token calls vividly illustrates the immense computational burden and the critical need for high-performance chips, a need that currently only Nvidia's technology can consistently meet for advanced applications.
Key points
- Chinese AI firms face compute constraints for large-scale AI deployment due to reliance on Nvidia chips for complex tasks.
- Domestic chips can handle low-quality inference, but high-tier applications like coding still require Nvidia processors.
- The demand for high-quality tokens far outstrips supply, creating a 'bipolarisation' in the market.
- China's average daily token calls exceeded 140 trillion in March, a 1,000-fold increase from early 2024.
- Optimizing software is a key strategy for Chinese companies to cope with surging inference demand and limited chip supply.
Chinese AI companies are actively optimizing software to improve the efficiency of existing hardware, which could help stretch the utility of scarce high-end chips and potentially enable domestic processors to handle a wider range of inference tasks over time. This focus on software innovation offers a pathway to mitigate immediate hardware constraints and enhance the overall performance of China's AI infrastructure.
The persistent reliance on Nvidia chips for high-tier AI tasks, especially coding, highlights a significant vulnerability for China's AI sector amid restricted access to advanced processors. Domestic chips are currently limited to low-quality inference, making it difficult for Chinese firms to find viable commercial paths for their advanced AI models and potentially hindering their global competitiveness.
Market signals
- NVDA Chinese AI firms still require Nvidia chips for high-tier tasks like coding, ensuring sustained demand for Nvidia's advanced processors despite efforts to use domestic alternatives.
AI-generated analysis of potential market relevance. Not financial advice.
