Sam Altman says personal AI agents are about to become AI’s killer feature

A number of moments from my brief sit-down with Sam Altman brought the OpenAI CEO’s worldview into clearer focus. The first was when he pointed to an iPhone SE (the one with the home button that’s mostly hated) and said, “That’s the best iPhone.” More revealing, though, was the vision he sketched for how Artificial Intelligence (AI) tools will become even more enmeshed in our daily lives than the smartphone.

“What you really want,” he told MIT Technology Review, “is just this thing that is off helping you.” Altman, who was visiting Cambridge for a series of events hosted by Harvard and the venture capital firm Xfund, described the killer app for AI as a “super-competent colleague that knows absolutely everything about my whole life, every email, every conversation I’ve ever had, but doesn’t feel like an extension.” It could tackle some tasks instantly, he said, and for more complex ones it could go off and make an attempt, but come back with questions for you if it needs to.

The Future of AI, by Matthew Griffin

It’s a leap from OpenAI’s current offerings. Its leading applications, like DALL-E, Sora, and ChatGPT - which Altman referred to as “incredibly dumb” compared with what’s coming next - have wowed us with their ability to generate convincing text and surreal videos and images. But they mostly remain tools we use for isolated tasks, and they have limited capacity to learn about us from our conversations with them.

In the new paradigm, as Altman sees it, the AI will be capable of helping us outside the chat interface and taking real-world tasks off our plates.

When asked if we’ll need a new piece of hardware to get to this future Altman had this to say ... Though smartphones are extraordinarily capable, and their designers are already incorporating more AI-driven features, some entrepreneurs are betting that the AI of the future will require a device that’s more purpose built. And some of these devices are already beginning to appear in. There is the (widely panned) wearable AI Pin from Humane, and the Rabbit R1, for example - Altman is an investor in the company but has not exactly been a booster of the device. He is also rumored to be working with former Apple designer Jony Ive on some new type of hardware.

But Altman says there’s a chance we won’t necessarily need a device at all.

“I don’t think it will require a new piece of hardware,” he said, adding that the type of app envisioned could exist in the cloud. But he quickly added that even if this AI paradigm shift won’t require consumers buy a new hardware, “I think you’ll be happy to have [a new device].”

Though Altman says he thinks AI hardware devices are exciting, he also implied he might not be best suited to take on the challenge himself: “I’m very interested in consumer hardware for new technology. I’m an amateur who loves it, but this is so far from my expertise.”

Upon hearing his vision for powerful AI-driven agents people wondered how it would square with the industry’s current scarcity of training data. To build GPT-4 and other models, OpenAI has scoured internet archives, newspapers, and blogs for training data, since scaling laws have long shown that making models bigger also makes them better. But finding more data to train on is a growing problem. Much of the internet has already been slurped up, and access to private or copyrighted data is now mired in legal battles.

Altman is optimistic this won’t be a problem for much longer, though he didn’t articulate the specifics.

“I believe, but I’m not certain, that we’re going to figure out a way out of this thing of you always just need more and more training data,” he says. “Humans are existence proof that there is some other way to [train intelligence]. And I hope we find it.”

An example of what he’s talking about there is, of course, Zero Shot or Few Shot Learning where AI’s either require small amounts of data or none atall to learn new tasks and things “intuitively” in much the same way animals intuitively learn to walk without specifically being taught what it is or how to do it.

OpenAI’s central vision has long revolved around the pursuit of Artificial General Intelligence (AGI), or an AI that can reason as well as or better than humans. Its stated mission is to ensure such a technology “benefits all of humanity.” It is far from the only company pursuing AGI, however. So in the race for AGI, what are the most important tools? Altman was asked if he thought the entity that marshals the largest amount of chips and computing power will ultimately be the winner.

Altman suspects there will be “several different versions [of AGI] that are better and worse at different things,” he says. “You’ll have to be over some compute threshold, I would guess. But even then I wouldn’t say I’m certain.”

You thought he’d answer that? When another reporter in the room asked Altman if he knew when the next version of GPT is slated to be released, he gave a calm response. “Yes,” he replied, smiling, and said nothing more.