|
Getting your Trinity Audio player ready... |
Karen Hao, however, already knew the story well. In fact, watching the trial she felt vindicated. “It was good to see lots of what I had discovered being laid out,” she tells me when we meet in London a few days after the trial ends.
Hao was given unprecedented access to OpenAI’s offices in 2019, and has since spoken to hundreds of former employees and people within Altman’s inner circle to piece together the story of how OpenAI went from an ideological non-profit, with the purpose of “saving humanity”, to an engine of record financial investment and controversy (last week The Wall Street Journal reported it was preparing to file for a public listing, expecting a valuation in excess of $1 trillion). It is the central theme of her book, Empire of AI: Inside the reckless race for total domination, published last month, in which she argues that the maker of ChatGPT sparked a race for technological progress which is rapacious, extractive and bad for humanity.
Musk lost his lawsuit, in which he accused his former partner of breaching a non-profit contract by shifting OpenAI to a commercial enterprise. But the trial was a distraction anyway, says Hao. What she really wanted people to pay attention to was how we got here, and what we need to do next.
When Hao first arrived at OpenAI’s offices in San Francisco in 2019, the company’s chief technology officer greeted her with a tentative smile. “We’ve never given someone so much access before,” Greg Brockman told Hao, inviting her to embed herself in their company for a profile piece she was writing.
Two weeks earlier, on July 22, 2019, OpenAI had received a $1 billion investment from Microsoft. For Hao, a young technology reporter who had studied engineering and worked for a Silicon Valley start-up before joining the magazine MIT Technology Review, it was a career-defining opportunity. It was also the moment that the scales fell from her eyes. “Right off the bat, I started realising that something was not right,” says Hao, 32.
Hao had become a bit of a cynic after spending too much time around Silicon Valley start-ups masquerading as noble endeavours. But OpenAI, which had been founded in 2015, was supposed to be different. It was a non-profit that had dedicated itself to developing artificial general intelligence (AGI), the most powerful form of AI yet conceived. The idea was that this superintelligence could replicate, and then surpass, human intelligence. In OpenAI’s telling, it could become powerful enough to destroy the world, or to create global utopia, solving problems humanity wasn’t smart enough to.
They wanted it to become the latter, and OpenAI would be a transparent and collaborative tool to enable the world to get there.
Hao was surprised to find, then, what she considered a distinct lack of transparency. She was chaperoned everywhere. She wasn’t allowed to visit certain floors or attend certain meetings. “As I was talking to researchers, I noticed that they kept being very nervous about saying things they weren’t supposed to, which was bizarre, because the entire premise of OpenAI was they were going to share everything,” says Hao.
A security guard was given a picture of her face and told to be on the lookout if she appeared unapproved on the premises. Employees were warned on Slack not to speak to Hao beyond “sanctioned conversations”. The atmosphere was “competitive, secretive and insular”.
Speaking to insiders later on, she would hear how what had begun as an organisation throwing ideas at the wall “to see what stuck” had been transformed under Altman’s singular obsession: to achieve AGI before everyone else. This included competitors, such as Google, but also states, such as China.
Its scientists and researchers were some of the brightest minds in the industry. But, Hao says, their belief in AGI was something more akin to a religious fervour. She calls it “the ideological pursuit of the machine god”.
Several former OpenAI employees told Hao about a retreat in the hills of the Sierra Nevada mountains where senior scientists, dressed in bathrobes, sat around a firepit at a sprawling lodge and watched as Ilya Sutskever, OpenAI’s brilliant and eccentric chief scientist, burnt an effigy “representing AGI”. Friends who joined the company described to her how it was only after leaving that they “came back down to earth”.
Hao claims that there was paranoia about company secrets being leaked and spies getting in. Sutskever once mused to colleagues about what he ought to do if his hand were cut off to be used in a palm scanner for unlocking OpenAI’s secrets, Hao writes in her book. He proposed building a secure containment facility, a “bunker” within which would be a computer totally disconnected from any network. Another executive, Dario Amodei, used a disconnected computer to write critical strategy documents, connecting it directly to a printer so he could circulate only physical copies.
What most disillusioned Hao, however, was how this relentless pursuit of superintelligence was moulding the company and the industry.
OpenAI decided that the best way to achieve AGI was to take its large language models (LLMs) and dramatically scale them up. This meant “pouring ever more data into them and training them on supercomputers larger than anyone has ever built in human history”, says Hao. All this new processing power cost money, and OpenAI began a for-profit arm in order to find it.
“It’s hard to overstate how much this idea of scale was considered a scientific extreme at the time,” says Hao, who describes it as a “brute force” approach.
Until this point, AI research had been much more targeted. Scientists used small and limited data sets to test hypotheses about what artificially intelligent machine learning could do, such as detecting signs of Alzheimer’s by feeding it datasets of brain scans.
Now, it was about feeding as much data to the AI as possible in the hope of it developing “intelligence” in any field. The results were increasingly fluent models which seemed impressive, although some were sceptical whether this represented novel problem-solving and true intelligence. Nonetheless, OpenAI’s competitors raced to upgrade or create their own LLMs.
“We’ve seen this collapsing of the entire AI field and the entire industry towards a singular approach that is intellectually extremely lazy and societally deeply harmful,” argues Hao.
“All of the things that we see in terms of the negative impacts of AI come from this scaling idea.”
The vast, water-consumptive data centres popping up all over the world, driving up local energy prices, are a result of AI companies feeding LLMs more and more data in an effort to expand their knowledge base. Meanwhile, companies trawling the internet for morsels of information on which to train them has eroded our privacy and intellectual property, Hao argues.
In the beginning, OpenAI focused on “clean data” — authoritative texts such as peer-reviewed research papers, she says. But they quickly ran out. So they began scraping the web for the dirty stuff, hoovering up data from sources such as social media and online forums. This could be anything from a review of a restaurant to someone arguing that the Earth is flat on Reddit. This poor quality data created poor quality results — flawed chatbots spitting out half-truths and conspiracies.This may paint a gloomy picture, but Hao is adamant that she is not an AI “doomer”. The answer to all this, she says, is not that we should stop the progress of AI, which would mean shutting down a world of possibilities.
“We could still rapidly advance AI, just in a different direction,” says Hao. We need to develop more targeted AI. This wouldn’t require huge reserves of computing power, or massive amounts of indiscriminate data. It would simply mean having more specific aims, such as using AI to discover new drugs by feeding it small and specific datasets, and testing the results with rigorous scientific research. It might not create something superintelligent, but you couldn’t argue it wouldn’t benefit humanity, says Hao, and without the unintended consequences.
Ultimately, for her, it all boils down to the question she asked OpenAI’s executives in 2019 on her first day in the office. It was one which they struggled to answer, says Hao, and which, seven years later, she is still asking: why choose to aim for human-surpassing superintelligence at all?
Karen Hao’s book is, “Empire of AI: Inside the reckless race for total domination,” is published by Penguin
