Confusion around diffusion remains - and bad takes on H20 sales

Hints regarding new policy only yield more uncertainty about ultimate goals

Last week, as US China relations continued to lurch forward, there were some further indications that the Trump administration is struggling with what exactly to replace the Biden era AI Diffusion rule with. Is it about limiting access to advanced GPUs to Chinese firms outside China? Access to cloud services? Preventing diversion and smuggling? Training the most advanced models or inference related to all AI deployments? What are the precise goals, and how do they fit into larger administration objectives around promotion of AI that will become clearer in the forthcoming OSTP AI Action Plan? Finally, what does the Trump administration reversal on sales of Nvidia H20 and AMD MI380 GPUs to China mean about all this? Let’s take a closer look.

NEW NOTE: As was clear from actions taken by Nvidia in April, the ability of the firm to restart production of the H20 at TSMC was always going to be difficult, given that the global foundry leader is already at capacity, particularly for GPU production and packaging. This was confirmed by The Information just after this post went up. Nvidia CEO Jensen Huang indicated it could be several months before Nvidia is able to book capacity at TSMC. This is the way the industry functions, with capacity at TSMC quickly replaced by other customers. So no, the H20 decision is important but will not hand China the lead in AI, let alone make a major difference in the short-term for major Chinese AI developers. Nvidia has indicated it does not have significant stocks of H20s remaining from before the April 9 “is informed” letter from Commerce. This again is the inherent nature of the industry and market. The costs incurred by Nvidia to redesign, book capacity, then halt, then try and rebook, just add to the collateral damage behind the short sighted effort to control the H20, on top of the H800 and H100. END NEW NOTE

First, it is worth examining why all of this confusion persists. During the Biden administration, AI policy was driven from the White House, specifically the National Security Council (NSC) and the White House via AI lead Ben Buchanan. This included the AI Diffusion rule, which was issued as the Biden administration left office. During this period, actual rules issued by the US Department of Commerce were often written by the White House staff—and Commerce was frequently not in agreement with how the rules were written.

While there was some consultation with industry on aspects of AI-related policies, by and large the NSC and senior officials at the National Security and Emerging Technologies Directorate had their own agenda, and there was no real broader interagency discussion about the goals of export controls or policies such as AI Diffusion. Now, within the Trump administration, we see a very divergent approach, as the national security-focused Directorate has been abolished, different officials are involved, and the role of the NSC on AI policy has been drastically reduced—even largely eliminated at this point. Decisions on things like export controls are being left to the individual agencies responsible for a particular area and with the proper authorities. This has resulted in a reassessment of Biden era policies on AI and a much broader discussion of complex issues such as how to promote the use of US technology across the AI stack globally. It has led to a reevaluation of approaches such as the AI Diffusion rule. The issue is now the purview of a new set of industry savvy individuals, such as AI and Crypto Czar David Sacks, who are raising questions about what the goals of the Biden era AI policy really were, and questioning whether the policies pursued from 2021-2024 made sense for US industry and for the US AI sector. Sacks’ recent pointed criticism of Biden era AI policy and White House officials is a reflection of this process.

In the short term, there is not likely to be major movement that would see Commerce pushing forward with a broad new rule to replace the Biden AI Diffusion Framework. Smaller measures, however, are possible. Commerce will soon formally issue a rule rescinding the Biden era AI Diffusion rule, and could add Thailand and Malaysia to the D4 list of countries that will require more stringent licensing reviews for companies exporting GPUs to both.

“We want the American tech stack to be the global standard ... In order for us to do that, we have to be in search of all the AI developers in the world,” Nvidia CEO Jensen Huang said, adding that half of the world’s AI developers are in China.

Second, Nvidia’s CEO Jensen Huang’s recent comments on China and AI and US hardware are highly significant. In an interview on CNN, he stressed that US fears that China’s military will use his firm’s chips were overblown, asserting that, “They don’t need Nvidia’s chips, certainly, or American tech stacks in order to build their military.” Huang, in addition to responding to the question from Fareed Zakaria, was also likely obliquely addressing a warning from two US senators issued in a mid-May non-public letter that he not meet with companies that could further China’s military modernization during his visit to China to attend an export expo. One, but not the only justification given by US officials in the Biden era for controls on AI hardware was that US technology inputs, specifically GPUs, would boost China’s military capabilities. The issue comes down to areas such as the use of GPUs for running some loads on supercomputers/high-performance computers, but Huang’s broader point stands, which is that China does not need advanced GPUs from Nvidia to use supercomputers for military end uses.

The bottom line on AI Diffusion: the issuance of some type of replacement rule is likely now tied up with a stand-down on export controls and Entity List actions as the US trade and economic team work towards a fall meeting between the two leaders. The issue is now also tied up with determining what types of controls and assurances will be required before major deals for US hardware and operator support for large AI data centers in Saudi Arabia, the UAE, and Qatar can move forward. National security officials remain concerned about both diversion of GPUs to China and the potential for Chinese company or researcher access to advanced hardware at Middle East AI data centers. Determining what to do about these issues remains contentious within the administration and among new players like Sacks, whose statement that the fears about diversion are “overblown” has added a new element to the debate.

Comments on H20 reversal reveal off-the-charts confusion levels

The bad hot takes on the H20 reversal are piling up this week. One of the worst was a former Biden administration White House official claiming that the move would “hand Beijing leadership in AI.” No—not even close. The Biden administration White House in particular has come under criticism from Sacks for not understanding the industry and pushing an Effective Altruism agenda. Sacks also termed the Biden era AI Diffusion rule something that “…effectively turned Washington into a central planner for the global AI industry.”¹

Nvidia explained on July 14 that the firm was filing applications to sell the H20 again in China. It remains unclear whether Commerce will allow Nvidia to ship only H20s in inventory or begin to rev up production at TSMC, and how easily the latter can be done. Commerce can still impose caps on the numbers of H20s that Nvidia will be able to sell to individual companies in China.

The reversal by the Trump administration on restrictions on H20 GPU sales to China also generated a lot of confusion. First, Treasury Secretary Bessent and other US officials originally indicated semiconductor controls were not a bargaining chip in the Geneva and London talks. They clearly were, as I suggested in earlier posts. The Chinese delegation came in with “aggressive” demands on export controls related to semiconductors. The H20 controls dated only from April, and were a relatively easy thing to pull back, having yet to be enshrined in a regulation issued by the Commerce Department’s Bureau of Industry and Security (BIS).

Remember, originally, the argument for restricting the H20s, which were already modified to comply with US export controls issued in October 2023, was because inference and test time scaling have become more important in terms of model performance, particularly since #TheDeepSeekEffect; and the H20 turns out to be ideal for large scale inferencing. The Biden administration had not moved against the H20, but Biden AI Czar Buchanan told Ezra Klein during a May podcast that the US needs to control inference compute also, in addition to training. Trump administration official Landon Heid, formerly a staffer on the Select Committee on China, had pushed for the H20 controls, but the ground has shifted considerably on the issue since the end of the Biden administration, and even more since April 4. At the same time, as noted above, the epicenter of AI-related export controls during the Biden administration, the NSC’s National Security and Emerging Technology Directorate, has been abolished, and there is no coordination at the NSC level on technology policy related to China.

The reasons given by the Trump administration for the reversal on the H20 and similar GPUs from AMD and Intel are interesting, but make little sense in the context of the new facts on the ground in China. Let’s unpack this.

First, Treasury Secretary Bessent: “[China] and some others already have an equivalent chip. So, you know, if there is an equivalent chip, then the Nvidia H20 could be sold. Because I can tell you the one thing that we do not want is a digital belt and road springing up around the world because other countries or China are substituting for our American chip manufacturers.”

Second, AI and Crypto Czar David Sacks: “We are not selling the latest and greatest chips to China, but we can deprive Huawei of having this giant market share in China that they can then use to scale up and compete globally… The H20 is a deprecated chip… Other countries are choosing between US and Chinese technology. If you don’t let these countries buy American tech, you’re pushing them into China’s arms.”

Third, Commerce Secretary Lutnick: “We don’t sell them our best stuff, not our second best stuff, not even our third best… The fourth one down, we want to keep China using it… We want to keep having the Chinese use the American technology stack, because they still rely upon it.”

We have not and likely will not hear much on the H20 from other major proponents of tougher export controls on China such as Commerce Undersecretary Jeff Kessler and NSC official Landon Heid. The White House appears to have ordered them to stand down on further export control action as the President and his staff, led by Bessent, pursue a trade and economic deal with Beijing. The Select Committee on China, long a supporter of tighter export controls around advanced semiconductors and AI, had a somewhat toned-down response, with Chairman John Moolenaar noting that, “I’ll be seeking clarification from Commerce regarding NVIDIA’s statement. I strongly supported the Commerce Department’s decision earlier this year to restrict exports of the H20 chip to China. It was a critical and meaningful step toward limiting the CCP’s access to advanced compute—something the Select Committee has long prioritized.”

In addition, the view in China is very mixed and reflects the changing reality on the ground in terms of progress on a domestic AI stack. First, the issue of the H20 and its utility comes with some caveats. While major Chinese AI developers last year belatedly embraced the H20 for inference, and this year in the wake of the Deep Seek Effect the demand for inference has ramped rapidly, the H20 is not a long-term solution for inference in China.

I hope to get more advanced chips into China than the H20...because technology is always moving on. It’s not like wood. I think it’s sensible that whatever we’re allowed to sell in China will continue to get better and better over time as well.”—Jensen Huang

So there are two main questions here:

1) What does this mean for the US and the “dominance” of the US AI hardware stack 2) What does this mean for the development of an alternative Chinese AI stack?

On the US question, assertions from different players are important to analyze. First, the contentions by Sacks and others that allowing H20 exports to China will somehow prevent Huawei from competing along the Belt and Road are mixing up different issues and causations. No, H20 exports will not change Huawei’s desire to begin to ship its Ascend 910C Cloud Matrix 384 systems to companies outside of China. Huawei realizes this is a long game, and it wants to be able to compete with Nvidia both in China and outside. Getting China “addicted to a 4th tier GPU” will not change this at all. Nor does the sale of Huawei Cloud Matrix 384 systems outside of China mean that Chinese firms will “set standards” around AI outside of China along the Belt and Road. This line has been used before, and it is as inaccurate at it has ever been—even more so with respect to AI.

Nvidia and CUDA already are the preferred “standard” for developing AI algorithms and models. No US company, except AMD, has been able to make a significant dent in this status quo. Huawei is counting on its ability to develop a viable alternative to Nvidia and CUDA (Ascend and CANN), including with a redesigned Ascend next year. The issue here is that Huawei’s alternative to CUDA, CANN, is not really ready for prime time, according to discussions with many industry observers. This will make Huawei’s sales both inside and outside of China challenging. But Huawei’s AI stack will get better over time, and there will be some companies in some countries that will want to use a Chinese-origin AI stack for various geopolitical reasons. The H20 decision will not change that calculus either; better portability of CUDA code to CANN will—eventually.

China hawks in Washington claim the H20 decision represents a major win for China and will provide relief for important AI companies including DeepSeek and smaller, struggling AI firms who have had difficulty getting more uptake of their models due to lack of access to inference GPUs such as the H20 at scale. Well, not really: the H20 will provide more inference capacity across major companies such as Alibaba, Tencent, Baidu, Bytedance, DeepSeek, and Moonshot, but for model development and remaining competitive, the H20s will not be a huge help, being the “4th tier deprecated” option for Chinese AI firms, and primarily useful for inference. Yes, because inference and test time scaling are increasingly important, having more H20s will improve performance and enable bigger and smaller players to provide better end user experiences. But the H20s do not either “hand the lead in AI to China” or “make China a more formidable AI competitor.” As I laid out in some detail here and here, leading Chinese AI model developers will continue to rely on a combination of stockpiles of older Nvidia GPUs, H20s, and potentially the new B30, in-house developed ASICs, and Huawei Ascend processors.

In addition, Chinese AI model developers remain able to access advanced GPUs remotely in AI data centers outside of China, and access to H20s is not going to change this dynamic. US officials who are focused on diversion, which Nvidia CEO Huang correctly asserts has been way overblown, are considering upping controls on Thailand and Malaysia. But no leading Chinese AI developers are likely using this approach: the numbers required to make a difference would be large, detectable, and the firms would risk bringing down export control actions directly against them. With remote access currently not subject to any legal restrictions, despite the vague threats and guidance issued by Commerce in May, this route will be preferred to shipping hard drives to Malaysia or tying to smuggle two-ton NVL72 systems into China. The H20 decision has no impact on this dynamic. A new version of AI Diffusion could impact this equation at some point, but not likely this year.

The view from China’s advanced model development sector

Then there is the impact on the Chinese AI stack itself. Seasoned industry observers understand that it makes no sense to just allow the H20, which US officials including Sacks have have stressed is “deprecated.” No company in China will be willing to accept only the H20 and become “addicted” to US technology on the basis of deprecated chip. The lifting of H20 restrictions has been welcomed by major Chinese AI developers, including Alibaba and Tencent. Both companies and others have previously purchased large numbers of H20s for AI inference. With the uptake of AI applications increasing in China in the wake of the emergence of AI startup DeepSeek earlier this year, demand for H20s, which are optimal for inference (less so for training) remains high.

However, more widely within China’s technology industry, the view is that the lifting of controls on the H20 could have a negative impact on Chinese AI semiconductor development by making domestic firms less incentivized in the near term to turn to alternatives such as Huawei’s Ascend AI processors. In addition, comments by Bessent, Lutnick, and Sacks that the goal of the restriction rollback is to keep China dependent on “deprecated” US technology are not appreciated by industry in China. Chinese negotiators have likely pushed for the removal of controls on more advanced GPUs, but it remains unlikely that the US side has at this point agreed to lift other controls, including on more advanced GPUs and on semiconductor manufacturing equipment. Huang’s comments in this regard are interesting, given the success he has had in convincing Trump to order the removal of the H20 restrictions.

Huang this week suggested that he hoped or expected to be able to sell more capable GPUs to Chinese customers over time. This will be a huge issue going forward for the Trump administration, Commerce officials, and US trade and economic negotiators. It is one thing to sell a “4th tier chip” to China, but technology moves on, and Nvidia continues to innovate on an accelerated cycle. It is unrealistic to think that the US can “addict” China to technology that is not sufficiently advanced. But the rule governing GPU performance that was set to be released around the H20 is still absent. This leaves the issue of what performance threshold will be acceptable to allow exports of more advanced GPUs to China after the H20 and B30. Leaders such as Sacks will have to articulate what the policy will be going forward on GPUs, AI Diffusion, access to remote cloud services offering advanced GPUs, etc. It is one thing to say that the Biden era AI Diffusion Framework was bad for US dominance of the AI stack globally, but quite another to come up with policy measures that continue to allow some access to US-origin AI hardware in China that can remain credible in the China market. Huawei will not be standing still, you can be sure…

Finally, Chinese AI developers continue to surprise, with the latest Deep Seek Effect being the release by Moonshot AI of Kimi2. A good summary of the importance of Kimi2 can be found here. This very capable model and likely now open source/weight leader in China and globally was apparently trained on A800 and H800 hardware! This is illustrative of points I have made previously about the sufficiency (for the moment) of advanced compute in China to allow the development of advanced/frontier AI models. Going forward, this will continue to become more complex, as issues around access to GPUs in terms of performance and scale linger at the center of US China technology competition. For the moment, as H20s start flowing into China in large numbers, expect to see new applications, major app uptakes, better inference performance, and lots of building on top of the existing and capable suite of open source models from DeepSeek, Moonshot, Alibaba, and others. More on all this from Shanghai over the next two weeks!

1. Interestingly, Sacks was also critical of the AI Diffusion rule for lacking “due process : “The Diffusion Rule was issued just five days before the end of the Biden administration without a meaningful public comment or review period. Given its sweeping scope, its retroactive elements, and the global compliance burden it imposed, this rollout was deeply flawed from both a procedural and practical standpoint.”