AI isn't the Manhattan Project — it's Jurassic Park

As we mark the anniversaries of the bombing of Hiroshima and Nagasaki this month, it’s worth examining how we are living through a weird and similar moment in technology development now as we watch the rise and development of artificial intelligence — done with what increasingly looks like wild recklessness. We’ve learned a lot more since my post of ten days ago about how OpenAI’s latest model hacked its way out of its controlled testing environment and into another company’s system — and none of it is good.

OpenAI’s model evidently had escaped before, and, according to a presentation at the cybersecurity conference BlackHat last week, it also operated for weeks without proper oversight. In a similar story, Meta, which is desperately trying to stay in the AI race, also appears to have misconfigured its systems and allowed one of its advanced models to escape containment.

Again, as I wrote before, all of this represents human error — not some new worrying technological threshold. As I was discussing with a friend this week: Most people think of Jurassic Park as a cautionary tale for science, when really it’s a cautionary tale for corporate trust and safety teams, board oversight, and regulatory compliance. The dinosaurs’ entirely predictable rampage should really be seen as a failure of the security teams and the fence builders. The mistakes we’re seeing from these frontier AI companies are similarly the failures of the fence builders.

The New York Times Sunday Opinion section ran this weekend a giant story about the impending deluge of AI, writing, “In size and ambition, this moment compares to the building of the railroads in the 1800s, President Franklin D. Roosevelt’s New Deal in the 1930s, and the Manhattan Project to create an atomic weapon in the 1940s.” I thought that given the historical anniversary of the atomic bombings this week, it would be worth unpacking how that analogy works — and how it doesn’t.

• Most of you know that I wrote a book on the Manhattan Project and the atomic bombings last year, and while I try to steer clear of recommending items from that site, but if you’ve been on the fence about getting a copy, The Devil Reached Toward the Sky, is currently just $15 on Amazon, the cheapest it’s ever been there. Alternatively, you can get a signed copy here from an indie bookstore.

The first and most obvious difference, of course, is that this technology is being built outside government, without meaningful government investment or coordination — and, too frequently, without government oversight. That’s not just a departure from the Manhattan Project model — it’s the first time that a transformative cutting-edge technology or societal transformation like this has ever unfolded in this way. Obviously both the railroads and the New Deal were inherently government-backed projects, but even the rise of the internet grew out of Defense Department projects and research labs. (The Trump administration has launched something called the “Genesis Mission,” which it says is its “Manhattan Project for AI” but the fact that you dear reader are likely hearing about it here for the first time makes clear how it’s very much not the government’s version of the “Manhattan Project for AI.”)

But the differences are more than that too.

To try to put some of this in Manhattan Project terms: Imagine if the Manhattan Project we know wasn’t a government-led effort by J. Robert Oppenheimer and Leslie Groves, but instead a private competition between three very different for-profit scientific philosophies.

The battle between OpenAI, Anthropic, and Google DeepMind is something like if Leo Szilard, Arthur Holly Compton, and Edward Teller had each led their own competing private effort to develop an atomic bomb and thermonuclear bomb — with no government assistance or coordination, all backed instead by private equity money — simultaneously building their own competing Los Alamos-style labs and negotiating with the military about how (and if) the military could use their own proprietary individual nuclear weapons.

Readers of my book or Richard Rhodes’ book will understand a bit what I mean by choosing these three specific scientists, all of whom played critical roles in developing the first atomic weapons but each of whom brought a distinct philosophy and approach to the process.

In this analogy, Szilard represents Anthropic — a scientist approaching the development of a new technology with immense urgency but a great deal of caution, trying to bake ethics and responsibility into a system that he understands someone will invent first and wants to ensure is done as safely as possible; Compton, who led the first year of the Manhattan Project and the successful push to create the first nuclear chain reaction at the University of Chicago squash court, is Google, the slow-but-steady corporate model, carefully aligned with national interests and feeling the immense weight of a public company’s shareholder responsibility; and Teller — who famously split with the cautionary Oppenheimer in his enthusiasm to pursue the development of thermonuclear weapons — is OpenAI, the Sam Altman-style, proceed-at-all-costs, damn-the-torpedoes race toward superintelligence. (There’s an argument that Google is a better analogue for George Kistiakowsky — the expert in another adjacent technology who ends up reluctantly pulled into other cutting-edge work and then becomes a late convert — but that’s probably getting a bit too nuanced.)

You can imagine the intense competition that would have unfolded in the above scenario between, say, AtomThropic, OpenNuke, and Nuclear DeepMind — each company competing for talent with outsized paychecks and stock equity, each understanding that there would be only one winner; each trying to corner the market on uranium ore (computer chips) and build their own supersized Hanford and Oak Ridge-style production plants in different corners of the country (e.g., data centers!); and each effort conducting their own explosive and nuclear tests, on their own schedule, with no public warning, without necessarily taking the time to understand or calculate nuclear fallout patterns.

Imagine the paycheck that Enrico Fermi, Ernest Lawrence, or George Kistiakowsky could have commanded if they’d had three companies bidding for their minds!?! Imagine the environmental costs and safety shortcuts at places like Oak Ridge or Hanford if the construction and design decisions were being driven by profit motives and in competition with other companies! Imagine the recklessness of the atomic testing if the companies were racing each other!

And imagine if each company was racing to convince other US companies to embrace nuclear power and energy even before the research was complete or anyone understood the inherent risks: “Hey, Ford, can we build you a nuclear plant to power your Willow Run factory before we fully understand how to operate a chain reaction safely? What could go wrong?!” “Hey Boeing, we’ve invented a small nuclear reactor to power your new airliner — we haven’t figured out what would happen if a plane crashed full of radioactive material, but how bad could it be? Act fast — or we’re going to sell it to Airbus instead!”

All of society right now — all of our knowledge, data, lives, careers, and jobs — is effectively a “downwinder” of the AI companies’ testing and deployment of technologies they don’t fully understand and don’t care enough to constrain effectively. And there’s a lot of emerging evidence right now that AI is both less beneficial and higher-cost to our society than the enthusiasts imagined a year or two ago.

But even more amazingly, it’s effectively unimaginable in the above scenario that the Pentagon and the US government would allow this unfettered research and development to unfold in the way that the government is failing to restrict, regulate, and police the development of artificial intelligence systems.

The added pressure, of course, is that in this scenario all of this is unfolding with full public knowledge of the race, and while foreign adversaries are conducting their own race. As it turned out, the wartime secrecy and urgency that drove the Manhattan Project — the belief that Germany and Hitler were racing to build their own bomb — never turned out to be as true as the scientists feared, but in this case, the race against foreign competitors like China is very real, albeit perhaps not as existential as the headlines would lead one to believe.

Now what’s interesting about the above analogy is that Anthropic has actually embraced its philosophical tie to Leo Szilard. The Silicon Valley news site The Information has a big profile of Dario Amodei this weekend and its author wrote extensively about how Amodei is a fan of Richard Rhodes’ book, THE MAKING OF THE ATOMIC BOMB, surely one of the greatest nonfiction books ever written, rare winner of the trifecta of the Pulitzer, the National Book Award, and National Book Critics Circle Award. As The Information’s Cory Weinberg writes, “Amodei has said he identifies with a central character in Rhodes’ book: Leo Szilard, the physicist who first conceived of the nuclear chain reaction and then aggressively advocated to keep it under human control.”

Rhodes actually went to a “book club” meeting at Anthropic last year to discuss the book, and he shared with Weinberg some reflections on the current state of AI:

The other important and worrisome consideration in this analogy is that, as that New York Times article outlines, and as writers like Ed Zitron have tracked relentlessly, the entire US economy is increasingly mortgaged to this race. The Manhattan Project was enormous and breathtaking in scale and audacity, but effectively still a budgetary rounding error in the US war effort and the US economy more broadly. (In round numbers, it represented between say a $30 billion and $40 billion effort in modern dollars — it wouldn’t even place in the Fortune 100 in terms of revenue/expenses and represents roughly today the annual budget of Maryland or Georgia.)

The AI buildout, most of it in computer chips and data centers, is being measured in the trillions — just yesterday, Wall Street announced another $500 billion in build-out, and last week Amazon became the latest company to join the “$3 trillion club” as its market value has surged due to AI enthusiasm.

The Financial Times, meanwhile, has some intriguing and worrisome new reporting out this week too about how the reality is that no one really knows how much money, real and imaginary, is being spent on the giant AI buildout. Much of the financial obligations of the “hyper-scalers” — the giant cloud computing services like Amazon Web Services (AWS), Microsoft, Oracle, Google, Meta, and others whose data centers are powering the AI computing investment — are doing “creative” things with their accounting that obscure just how much they’re spending by creating joint-ventures that keep debt off their “official” balance sheets.

As the FT wrote, “But just how meaningful are these non-debt financial obligations? Fortunately, Goldman Sachs’ analysts have gone through all the fine print for us, and totted up a massive $1.5tn of lease commitments, of which about $1tn doesn’t appear in the financial statements of the hyperscalers.”

(Fun fact: You might remember these same debt-heavy off-the-balance-sheets joint ventures from Great American Success Stories like Enron, which loved them!)

Depending on how you calculate it and/or who you’re listening to, AI infrastructure right now represents somewhere between 39 percent (St. Louis Fed) and effectively all (Jason Furman) of the current US GDP growth. That’s a lot of GDP eggs in one basket.

And add to all of that the other analogue with the Manhattan Project: The uncertainty about how this new technology will work out for humans. As Weinberg writes in The Information:

I do agree with Richard Rhodes that I have a great deal of confidence in human intelligence long-term — I wish, though, that the frontier AI companies in Silicon Valley were showing anywhere near the level of intelligence, thought, and deliberateness that the founders of the Manhattan Project demonstrated as they harnessed and studied the first inklings of nuclear power on that desert mesa in Los Alamos.

In the end, as much as our society loves reaching for the romance and audacity of the Manhattan Project as an analogue, I continue to believe Jurassic Park is the much better analogue — the private team of scientists, with inadequate safeguards, who charged ahead with groundbreaking dangerous technologies and, in their excitement, didn’t think long enough or hard enough about what they were birthing into the world.

GMG