The core case for export controls on China AI isn't mil civ fusion, CBRN, or cyber—it's AGI/AMI, and DSA
Official explanations fall short, but the real reason behind AI controls—and DeepSeek’s role—remains unexamined
As part of a very good but at times confusing and scattered recent interview, Biden administration AI czar Ben Buchanan responded to Ezra Klein’s question about what a world where China gets to Artificial General Intelligence (AGI) first looks like by emphasizing increased state capacity in China for cyber operations as the primary risk. (China already has CBRN weapons of course, so this important but less relevant national security risk could not be trotted out.) Buchanan also admitted that advanced AI would mean new advantages for cyber defense, among them better code writing, better defenses, and more intrusions detected. The Biden White House under Buchanan’s lead did a lot of important work on AI security issues and significantly advanced the dialogue on risks around this important technology. But the China piece of this effort has been fraught with some problematic assumptions and characterizations (see, for example, the confusion around AI Diffusion).
AGI refers to a type of artificial intelligence capable of understanding, learning, and applying knowledge across a broad range of tasks at a human level or beyond, not limited to specific or narrowly defined functions. AGI would possess the flexibility, adaptability, and reasoning ability similar to human intelligence. — ChatGPT 4.5
The implication of Buchanan’s characterization appeared to be that if a Chinese private sector company, say DeepSeek or Alibaba, achieves something akin to the ill-defined but often-used term AGI1—a term that Buchanan said he didn’t like (I prefer advanced machine intelligence or AMI)—the first thing Beijing would do with this capability is launch a crippling cyber attack on the US. Which raises the question: is this really the threat of China’s “winning” the race to AGI? Is it perhaps the other fear Buchanan mentioned, that AGI would enable a more perfect “surveillance state” in China? Both of these notional outcomes seem woefully inadequate to account for Klein’s, Buchanan’s, and the AI labs framing this debate’s insistence that AGI/AMI will be the most transformative technology of all time. Klein astutely pushed back on US policy here, noting that if the technology were really so game-changing, why was it that all the Biden administration came up with was export controls? What about the societal impact, job losses, loss of human dignity, etc.? And what about China? Would China not see export controls in this context as an act of war?
The debate around what AGI/AMI2 means is hot among leading AI researchers and companies. While critics like Gary Marcus hold that recent talk of impending AGI/AMI by senior officials at leading labs like Anthropic is hype—and podcasters and reporters are not pushing back—he sees that hype as mainly the product of people who believe in the magic formula of neutral networks and scaling, because “neural networks alone are not enough to get us to AGI”. But few, in fact, are claiming that scaling alone will enable the attainment of something we could call AGI/AMI.3
This debate highlights the uncertainty around AGI/AMI that some in the US government appear to underestimate, perhaps because they talk chiefly with labs primarily pursuing scaling. This dynamic reinforces the belief within the US government that the approach of AGI/AMI warrants an all-out effort to prevent China from getting there first. Notably, US officials do not appear yet to include progress on embodied AI, including humanoid and other mobile robots, which are also developing rapidly and at scale—particularly in China.
Klein also pushed back on how China views US export controls, in the context of what Buchanan insisted is a race to AGI (and inference!) that “the US must win.” Buchanan insisted that the US could simultaneously talk to China about safety and put in even more stringent controls on the technology that would enable the US to win the race to the most consequential technology of all time. Klein was properly skeptical of this, as I have been for all of the nearly three years it has been an issue. In addition, and most significantly, Buchanan never described what the US winning the race to AGI/AMI would actually mean in concrete terms. Klein should have pressed him on this, because it’s the key to understanding what is really going on.
AGI, or Artificial General Intelligence, refers to a machine’s ability to understand, learn, and apply knowledge across a wide range of tasks at a level comparable to human intelligence. Unlike Narrow AI, which is designed for specific tasks, AGI can perform any intellectual task that a human can, with versatility and adaptability. — DeepSeek V3
Let’s examine this issue in greater detail.
Here are some of the points made by Buchanan:
The White House has been watching DeepSeek since November 2023. Buchanan asserted that “we read their papers and they have a very capable technical team.”
The media hype around DeepSeek is not warranted; DeepSeek innovations do not change the fundamental trend that favors more compute for training; and the White House was gratified by the comments from DeepSeek CEO Liang Wenfeng that he has been frustrated by DeepSeek’s inability to access more advanced GPUs. Buchanan asserted, without details, that DeepSeek had access to “smuggled” GPUs. (From my own significant inquiry into this, I believe it is unsupported by the evidence. Further, all major Chinese AI firms that I have spoken directly with insist they would not knowingly violate US export control laws by buying “smuggled” GPUs, due to their well-founded fear of landing on the Department of Commerce’s Entity List.)
The export controls remain very important. Buchanan and the other architects of the controls “do not regret the export controls for a minute.”
The US must get to advanced AI first. Buchanan seems to primarily define “winning” as China not getting there first.
The US must win both the race for training and for widespread deployment of AI applications supported by low-cost inference. Buchanan does not offer any explanation for why the latter is so important. (My view is that the vast majority of the inferencing application will be consumer and business use cases, and even with AGI/AMI, these applications will still be there.)
The Biden October 2024 National Security Memorandum on AI remains in effect, unlike the October 2023 Biden AI executive order, which the Trump administration has revoked. One of the NSM’s goals was to prevent a race to the bottom while advocating for cooperation with China and restricting certain national security-related AI applications. However, this approach was framed within a broader commitment to maintaining “U.S. preeminence.” (Presumably this means things like talking to the Chinese about not including AI in nuclear command and control, as happened during meetings on the margins of APEC in November 2024. This is a very low-hanging fruit if there ever was one.)
There is no contradiction in denying China the most advanced semiconductors—used to develop the most consequential technology humans have ever developed—and insisting on US preeminence, while expressing continued willingness to discuss AI safety with Beijing. Buchanan also asserted that Beijing should not get upset about this or see it as a major provocation. (Klein was politely incredulous at this assertion.)
It is important to note that these views were widely held in the Biden administration within the small circle of officials setting AI and export control policy, and by some DC think tanks which have acted largely as cheerleaders for these views. They continue to be shared by Trump administration officials and some DC think tanks, but they remain largely unexamined and unchallenged. Let’s examine them.
The day after: examining assumptions about the urgency of controlling China’s ability to develop advanced AI
The justifications for the first, unprecedented package of export controls focused on advanced computing, supercomputing, and their potential to aid China’s military modernization and alleged human rights violations.4 The language used does not mention either generative AI or frontier AI models. That would come later. Former Secretary of Commerce Gina Raimondo asserted in 2024 that the controls were designed to “prevent China from training frontier models.” But like the export control packages, Raimondo still linked the goal to preventing military use of advanced AI capabilities. There was no mention of getting to AGI/AMI first.
However, some time in 2024 things seem to have changed. One reason for the shift in thinking was the major progress in capabilities that the AI sector saw in 2024. No one was talking about AGI/AMI in October of 2023. By the end of 2024, everyone was—including think tanks in DC. In August, I participated in a closed-door debate about China and technology controls where one leading think tank representative, who was also was one of the architects of the use of export controls that led to October 2022, asserted that the real reason for the controls was that all the leading AI labs were close to developing virtual software engineers that could operate at the level of leading human AI developers. Instead of throwing hundreds of software engineers at AI development, these labs would soon be able to throw one million.5 That would accelerate the race to AGI/AMI—potentially very quickly. Hence the need to keep the gap with China as wide as possible to ensure the US got to AGI/AMI first.
This was not what people were saying in October 2022, before the release of ChatGPT-3, and before anyone had heard of DeepSeek. In just over 18 months, we moved rapidly gone from talk of GPUs being used to accelerate modeling of hypersonic missiles, to believing that getting to AGI first was in fact a much more fundamental game changer, with vastly broader implications than merely optimizing the development of more capable weapons systems, or even better cyber operations. So why did Buchanan choose to mention these latter, second-tier concerns when asked about what the world would look like if China got there first?
The answer is complex. It has to do with the fact that no one really understands how we will ascertain certain critical information, such as when precisely a particular company or group of companies has definitively reached AGI/AMI, and what the world will look like the day after. Will the difference be so marked that it will be obvious that the country where the technology was developed is a massive “winner”, capable of blackmailing or coercing other countries without companies with equivalent capabilities? No one knows—not even the theorists at RAND who have taken this up as the next major global issue, akin to nuclear weapons, deterrence, and strategic competition. However, they have come up with the concept of Decisive Strategic Advantage (DSA) to describe what happens when either China or the US gets to AGI/AMI first. Here, though, just as it is unclear what China would do if it achieved AGI/AMI and DSA first, it is not clear what the US would do with DSA. There would be some gap between how long the US held DSA, for example, and when China caught up. What would the world look like in each of these cases: China first to DSA, US first, or both achieving AGI/AMI in close proximity so that no one has DSA? Policymakers would have a tough choice in deciding what to do with this DSA capability while it lasted. Recognizing that you are in possession of DSA, understanding exactly where Chinese companies are in the process of catching up, and determining how best to use it toward some political ends, are very tough calculations indeed.
The mere possibility of such a singularity occurring, and the game-changing advantage that AGI/AMI would provide to the “winner”, is enough to drive proponents of this view to further tighten export controls, regulate AI model training through the Framework for AI Diffusion, and buy time to ensure that US firms rather than Chinese ones reach AGI first. Meanwhile, the DSA concept assumes that if the US achieves AGI, it must maximize the time before China catches up; and that when the US achieves DSA, it can maintain it for long enough to use it—but to what end, we do not know. One can only wonder at this reasoning.
This view is also responsible for the about-face that some, but not all, leading US AI labs have made on China, routinely issuing statements of support for the export controls. Recently the CEO of one leading US lab doubled down on what he sees as the China threat, basically channeling the US government view. He asserted in a recent interview that “whoever gets to AGI first will be dominant economically and militarily and everywhere else.” In addition, OpenAI’s submission to the Office of Science and Technology Policy (OSTP) for the US AI Action Plan included the accusation that DeepSeek was “state controlled”. As documented in this Substack and many other places, DeepSeek is about as far from “state controlled” as one can imagine for a firm in China, having been spun off and funded by a Chinese hedge fund. It has been clear to those closely following AI developments in China that DeepSeek was a completely new type of player in the AI space, and that bureaucrats in Beijing were just as surprised as those in Washington, and as tech bros in Silicon Valley, about DeepSeek’s capabilities given the lack of state backing. OpenAI here appears to be going beyond what the US government has asserted about DeepSeek, for unclear reasons.
Senior officials at a small number of leading US AI labs continue to assert, again without any evidentiary base, that DeepSeek had access to “export controlled or smuggled GPUs”. In fact, companies such as DeepSeek and all the leading Chinese AI labs have been very careful not to use export controlled or smuggled GPUs. All the evidence currently suggests that the GPUs that DeepSeek used, the A100s and H800s, were purchased when they were not export controlled. Why senior officials continue to repeat these accusations is troubling. In addition, these same senior officials are well aware of the dependence of the entire AI sector on TSMC and supply chains centered on Taiwan, but seem to see no disconnect between asserting the need to control the technology so the US can “win” and dominate China via AGI, and the fact that the entire basis for doing this lies in a territory Beijing considers to be part of China. Under the logic they are pushing, if Beijing detected that a US lab was close to AGI/AMI, it would be justified in choosing to blockade, disable, or bomb the fabs on Taiwan. That senior officials in Silicon Valley and Washington never acknowledge this rather major disconnect, as they assert that we are ever nearer to AGI/AMI, is also deeply worrying.
Governments around the world are just beginning to grapple with this issue. Some, like the UK, have done significant thinking about what the day after AGI/AMI is achieved could look like and how governments would need to respond. But this is an immensely difficult issue to think through, given the uncertainties around the technology, the pace of progress, and what the indicators will be that a capability has been developed that is instantly recognizable as game-changing. Explaining all this to policymakers, including those responsible for US China relations, is and will remain challenging.
As I have noted many times, the stakes here are enormous, particularly for Taiwan and TSMC, and perhaps more importantly, for global efforts to develop a framework around regulating advanced models and applications. If the US view is that everything must be done to ensure the US gets to AGI/AMI first, this has huge implications for both Taiwan within the context of US China relations, and efforts such as the Bletchley Park Process, designed to gain a baseline of agreement around things like cyber operations, CBRN, and autonomy. But if it is really about AGI/AMI, as I believe it clearly is, then this radically changes the context of these issues. Buchanan’s assertion that despite US technology policies and their justifications, Beijing should be willing to discuss AI security as seriously as CBRN and cyber operations, appears stunningly misguided. It seems the capacity to understand the implications of US policies is likely stronger in Beijing than it is in Washington. This is also a discomforting thought.
From The Information
Meanwhile, as I have been saying since coining #TheDeepSeekEffect, DeepSeek continues to focus squarely on AGI/AMI, and is the company most likely to be the primary contributor to reaching this threshold in China. DeepSeek has put a face on China’s efforts to reach AGI/AMI, and as Buchanan notes, the White House was hip to this early on. (One wonders if Buchanan’s team was also tracking Alibaba, Bytedance and other players with significant teams working on advanced AI.) The firm is not accepting outside investment, including from government-backed funds, and is not scaling up to expand commercial operations—this was never part of the business plan. DeepSeek has for the first time generated revenue from leasing its API, but is relying on third party partners for a lot of the inference capacity to run its models, while putting most of its effort into V4 and R2, likely to hit the streets soon to keep up the buzz. Given the focus on getting to AGI/AMI that is driving US controls, it is likely that we will see DeepSeek and other Chinese firms pursuing AGI/AMI added to US blacklists and subject to other US controls.
For Beijing this is also a dilemma. The success of DeepSeek has been due to its collaboration with the open source/weight community, and the ability of people like CEO Liang Wenfang to fly to the US and discuss AI development with industry leaders. Recently the firm has apparently buttoned down on the ability of some executives and engineers to travel, concerned about their security. This does not appear to be a Chinese government directive, as some have claimed, but likely reflects concern within the firm that individuals travelling to the US could be detained, have their laptops searched, and the like, given the hostile environment being created in the US for Chinese technology companies and STEM students and researchers. At present, Beijing does not appear eager to weigh in on this process, and Chinese government officials were not likely consulted before DeepSeek released both its open source/weight models and the detailed documents outlining its most important innovations. Whether this changes—and it very much could, particularly if the Framework for AI Diffusion is tightened significantly and all AI GPUs including H20 are banned for export to China—is more likely to be determined in Washington than in Beijing.
Having governments drive a wedge between US and Chinese researchers is something I cautioned against in 2020 when I spoke to the National Security Committee on AI (NSCAI), and this was included in their report. However, five years later, we have arrived at a different moment. The expectation of AGI/AMI in the near term is likely to put new pressure on collaboration between leading US and Chinese AI researchers and companies. If the open source model/weight community, and robust exchanges allowing innovation in this space, are now to be sacrificed on the altar of hysteria around the race to AGI/AMI and DSA, we all stand to lose. No one is talking about that either.
This article is based on extensive discussions in the US, Europe, and Asia on these issues with people close to all of the specific topics. I will be in China for the China Development Forum, Semicon China, and meetings with players across the AI stack in the coming weeks.
NOTES
In the new AAAI 2025 Presidential Panel on the Future of AI Research, the chapter on AGI notes: AGI is not a formally defined concept, nor is there any agreed test for its achievement. Some researchers suggest that “we’ll know it when we see it” or that it will emerge naturally from the right set of principles and mechanisms for AI system design. In discussions, AGI may be referred to as reaching a particular threshold on capabilities and generality. However, others argue that this is ill-defined and that intelligence is better characterized as existing within a continuous, multidimensional space. Some (e.g., [6]) contend that the lack of a clear definition makes AGI an unsuitable goal for AI research: human intelligence has many dimensions, and machines will likely far exceed humans in some areas while remaining inferior in others. Moreover, the criteria for comparison, including which particular humans serve as benchmarks and how much prior training they have received, are often left unspecified.
Depending on the definition of AGI/AMI one favors, it can also come with potentially major risks—including from sentience and autonomy—and societal downsides, which are seldom alluded to in the context of “getting to AGI before China” discussions. Also from AAAI report: For some, AGI represents a potentially dangerous “threshold” that we cross at our peril. As an example, the “Gladstone Report” [4] commissioned by the US State Department states that “AGI is generally viewed as the primary driver of catastrophic risk from loss of control.” Others use the term “transformative AI” [5] to cover AI systems that have the potential to cause massive disruption of human civilization, noting that this does not require full AGI. We note that sentience and autonomy are not part of core definitions of AGI, even if some have made implicit assumptions about AGI having these attributes.
ChatpGPT4.5 weighs in:
Prompt: So aren’t the arguments by Gary Marcus and others than AGI will not come from neural networks alone sort of a strawman?
ChatGPT said:
To some extent, yes, the argument that “AGI will not come from neural networks alone” can be a bit of a strawman, depending on how it’s framed. Here’s why:
1. Few Serious AI Researchers Believe “Just Scaling” Will Magically Create AGI
While some, like Ilya Sutskever and proponents of the Scaling Hypothesis, suggest that increasing scale could lead to AGI, most researchers acknowledge that neural networks alone in their current form are insufficient.
OpenAI, DeepMind, and Anthropic are all experimenting with memory, reinforcement learning (RL), external tools, retrieval-augmented models, and multimodal integration—showing that they do not think pure scaling alone will work.
The Scaling Hypothesis is often misunderstood—it doesn't mean “bigger = AGI” but rather that continued scaling will reveal emergent properties that may bring us closer to AGI.
2. Neural Networks Are Not “Just Stochastic Parrots”
Critics like Gary Marcus, Melanie Mitchell, and Yann LeCun (to some extent) argue that current LLMs lack real reasoning, planning, and causality. This is valid.
However, newer models show emergent reasoning, tool use, and in-context learning, suggesting that neural networks are not as rigid as critics claim.
Marcus often argues for hybrid systems (neurosymbolic AI), but many current AI research directions are already moving toward incorporating structured reasoning, retrieval-based memory, and planning—so the divide is less stark than he implies.
3. The Debate Is Really About What Needs to Be Added to Neural Networks
Not “if neural networks can lead to AGI” but rather “what extra components do we need?”
Critics often overstate the limitations of deep learning while ignoring how improvements (e.g., RL, external memory, multimodal learning) are addressing these gaps.
Even critics like Marcus acknowledge that neural networks will play a major role in AGI—they just argue that they need symbolic reasoning, structured memory, or other mechanisms alongside them.
So, Is It a Strawman?
🔹 In its strongest form, yes—because no leading AI lab believes in “just bigger neural networks = AGI.”
🔹 However, the criticisms still have merit—because neural networks alone (in their current form) do lack key properties like persistent memory, deep reasoning, and true world modeling.
The Real Question: Will AGI Be Mostly Deep Learning or a Hybrid System?
If emergent abilities keep improving with scale, we may need fewer "hand-crafted" components than critics assume.
If deep learning plateaus, then hybrid approaches (symbolic reasoning, retrieval-augmented models, world simulators) will be necessary.
Note, many of these justifications are either not accurate or misrepresent language used in Chinese documents. For example, the justification holds that China “has announced its intent to become the world leader in AI by 2030.” Close observers or technologists in the AI sector would find it risible that a government in 2017 could “announce” that “China”, via a specific company or companies, could become the world leader in AI by a specific date. In fact, the aspirational National AI Development Strategy, which I helped translate into English just after it came out in 2017, is a highly aspirational document that the Chinese drafters admitted to me in Beijing shortly after its publication was almost instantly out of date, given the rate of development of the technology. The translation of the phrase on leadership should be rendered, “become a world leader”, rather than “the” world leader, meaning that by 2030 the Chinese government hoped that some small number of Chinese companies would be globally competitive in some areas of the AI stack. Ironically, one of the leading AI model developers (GenAI was not on the radar in 2017 in China), DeepSeek, was funded and support not by the Chinese government, but by a Chinese private sector hedge fund, and has arguably already achieved the rank of a '“world leader” in development frontier AI models in 2025. So “China” is already there, government pronouncements or not. Meanwhile, US government officials have frequently asserted the goal of maintaining technology dominance over China across a range of technologies. Former National Security Advisor Jake Sullivan asserted in September of 2022 that “…computing-related technologies, biotech, and clean tech are truly “force multipliers” throughout the tech ecosystem. And [US] leadership in each of these is a national security imperative.”
From October controls: Implementation of Additional Export Controls: Certain Advanced Computing and Semiconductor Manufacturing Items; Supercomputer and Semiconductor End Use; Entity List Modification
Advanced computing items and “supercomputers” can be used to enhance data processing and analysis capabilities, including through artificial intelligence (AI) applications. The PRC is rapidly developing exascale supercomputing capabilities and has announced its intent to become the world leader in AI by 2030. These advanced systems are capable of sophisticated data processing and analysis that has multiple uses, and are enabled by advanced ICs. These systems are being used by the PRC for its military modernization efforts to improve the speed and accuracy of its military decision making, planning, and logistics, as well as of its autonomous military systems, such as those used for cognitive electronic warfare, radar, signals intelligence, and jamming. Furthermore, these advanced computing items and “supercomputers” are being used by the PRC to improve calculations in weapons design and testing including for WMD, such as nuclear weapons, hypersonics and other advanced missile systems, and to analyze battlefield effects. In addition, advanced AI surveillance tools, enabled by efficient processing of huge amounts of data, are being used by the PRC without regard for basic human rights to monitor, track, and surveil citizens, among other purposes. With this rule, BIS seeks to protect U.S. national security and foreign policy interests by restricting the PRC's access to advanced computing for its military modernization, including nuclear weapons development, facilitation of advanced intelligence collection and analysis, and for surveillance. BIS intends to impose controls on items subject to the EAR and U.S. person activities to limit the PRC's ability to obtain advanced computing chips or further develop AI and “supercomputer” capabilities for uses that are contrary to U.S. national security and foreign policy interests.