How AI Got Its Scary Story: Morning Coffee Reading

How Sam Altman and friends turned productivity tools into monster stories

This morning I was drinking coffee and reading Benjamin Wallace-Wells' piece in The New Yorker: "Can Sam Altman Be Trusted With The Future?"

It really connected with something I've been writing about, so I wanted to share a few thoughts.

The article tells the story of Sam Altman's rise, but that's not really the point. What's interesting is how much he's influenced the way media writes about AI. And how that shapes what the rest of us think about it.

The author walks through how Altman helped create this whole narrative that AI is going to replace us all, take our jobs, maybe even destroy humanity entirely. And why that story served OpenAI’s interests so well.

He also looks at why more reasonable voices—people like Microsoft's C.E.O, Satya Nadella, who talk about AI as just another ‘normal’ technology—don't get nearly as much attention.

After reading this New Yorker’s piece, you can’t help but wonder if all those scenarios about AI replacing us are nothing more than campfire ghost stories, thrilling to tell, but ultimately just meant to give us a little shiver?

For those who don’t have time to read the whole Benjamin Wallace-Wells' article, I’ll briefly share its key insights. This isn’t some AI summary, it’s just my own take on what I’ve found important.

The Term ‘AI’ Already Programs Us to Anthropomorphize Software

By reminding us when and by whom the term "Artificial Intelligence" was introduced into media discourse, the author highlights a crucial detail: the name itself gets us thinking about AI as some kind of mind rather than just software. And this has been true since their earliest days, long before they possessed anything close to their current capabilities.

“The “original sin” of this arm of technology … lay in a decision by a Dartmouth mathematician named John McCarthy, in 1955, to coin the phrase “artificial intelligence” in the first place. “The term lends itself to casual anthropomorphizing and breathless exaggerations about the technology’s capabilities”

The author then shows how and why the idea of all-powerful AI that will replace humans or pose an existential threat to humanity became the dominant story in the media. Sam Altman, Elon Musk and Sutskever's personal celebrity played a major role in this. In fact, much of the talk about AI threats and replacement was partly personal projection from people working in AI, and partly necessary “to attract investment and engineering talent.'‘ Nothing attracts investors quite like working on something that might change everything or end everything.

One view is that tech billionaires saw the brink early because they understood just how unequal—and therefore unstable—American society was becoming. But, inside the Valley, that anxiety often expressed itself in the language of existential risk. In particular, fears about runaway artificial intelligence surged around the time of the 2014 publication of “Superintelligence,” by the philosopher Nick Bostrom.

Elon Musk became fixated on an A.I. technologist, Demis Hassabis—a co-founder of DeepMind, which had recently been acquired by Google—whom Musk reportedly viewed as a “supervillain.” That same year, at an M.I.T. symposium, Musk warned that experiments in artificial intelligence risked “summoning the demon.”

Altman had been itching for a bigger project. The next Memorial Day weekend, he gathered hundreds of young Y Combinator protégés for an annual glamping retreat among the redwoods of Mendocino County. … standing before them, he announced that his interests had narrowed—from, roughly, all of technology to three subjects that he believed could fundamentally change humanity: nuclear energy, pandemics, and, most profound of all, machine superintelligence.

Therefore, “the deeper dynamic of contemporary artificial intelligence may be that it reflects, rather than transcends, the corporate conditions of its creation—just as Altman mirrored the manners of his Silicon Valley elders, or as a chatbot’s replies reflect the texts it has been trained on.”

Plus, as B. Wallace-Wells notes, “Altman is contending with a shifting cultural tide. Sometimes around 2016, the tone of tech coverage began to darken.” The scary AI story fit right into that mood.

AI Remains a Normal Technology: The Boring Truth?

On the other hand, voices arguing that AI is simply another technology—breakthrough, yes, but incapable of replacing humans—are becoming audible in the media too. The author quotes Microsoft CEO Satya Nadella from a recent interview:

"Nadella is now trying to shift the way the public thinks about A.I. by changing the way it's talked about—less science fiction, more office productivity. It's an uphill fight, and at least partly the industry's own fault."

“Nadella dismissed A.G.I. as a meaningless category. When Dwarkesh Patel, influential tech podcaster, pressed him on whether A.I. agents would eventually take over not only manual labor but cognitive work, Nadella replied that this might be for the best:

“Who said my life’s goal is to triage my e-mail, right? Let an A.I. agent triage my e-mail. But after having triaged my e-mail, give me a higher-level cognitive-labor task of, hey, these are the three drafts I really want you to review.” And if it took over that second thing? Nadella said, “There will be a third thing.”

Wallace-Wells writes “Nadella seemed quite convinced that A.I. remains a normal technology, and his instinct was to try to narrow each question, so that he was debating project architecture rather than philosophy. When Patel wondered if Nadella would add an A.I. agent to Microsoft’s board, a fairly dystopian-sounding proposition, Nadella replied that Microsoft engineers were currently experimenting with an A.I. agent in Teams, to organize and redirect human team members, and said that he could see the use of having such an agent on Microsoft’s board.”

But that's a tough sell. As the author writes:

"If Sutskever—who knew as much about the technology as anyone—could declare it 'slightly conscious,' it becomes markedly harder for Nadella, three years later, to reassure the public that what we're really talking about is just helpful new features in Microsoft Teams."

The author sees Sam Altman, Dario Amodei and Ilya Sutskever as still kids at heart in the tech world, boys who can’t resist telling monster stories. "Nadella is "the adult in the room, and adults are famously not so interesting”.