OpenAI starts to test AI that automates tasks on devices
OpenAI might very well be trying to create a new kind of device OS that uses agents to automate tasks and perform actions, and which could replace the need for apps.
Key takeaways
- Another type of AI Agent OpenAI is developing handles web-based tasks such as booking airfares or creating travel itineraries without access to APIs.
- AI agents differ from Robotic Process Automation (RPA), which still needs developers to manually code steps needed to complete a task, Luan said.
- In 2018, Google demonstrated that a computer can do things like call a hair salon to make an appointment for a user or contact a restaurant to book a reservation, all by itself.
Cite or link to this article
Griffin, M. (2024) 'OpenAI starts to test AI that automates tasks on devices', 311 Institute, 8 March. Available at: https://www.311institute.com/openai-starts-to-test-ai-that-automates-tasks-on-devices/ (Accessed: 25 October 2026).
OpenAI, the maker of ChatGPT, is now reportedly is working on Artificial Intelligence (AI) agents that can execute tasks for the user autonomously after The Information reported that one type of agent software OpenAI is developing would effectively take over a user’s device to automate complex tasks within an environment, such as the persons work. Normally, people will have to move the cursor, click and type to transfer between applications, but in this case, via a kind of Generative Operating System, ChatGPT could transfer the information on a document to a spreadsheet for analysis.
Another type of AI Agent OpenAI is developing handles web-based tasks such as booking airfares or creating travel itineraries without access to APIs. ChatGPT currently can do agent-like tasks, as we saw quite a while ago with the likes of AutoGPT, but it has to use the relevant third-party’s APIs.
The Future of AI keynote, by Futurist Matthew Griffin
Last November at its developers’ conference, OpenAI launched its Assistants API that lets users build agent-like experiences in their applications. But Adept CEO David Luan, who used to lead engineering at OpenAI, told The Information that many enterprise apps do not have APIs. Here, agents can fill in the gap, Luan said.
AI agents differ from Robotic Process Automation (RPA), which still needs developers to manually code steps needed to complete a task, Luan said. In contrast, AI agents can do more complex, unstructured work with little guidance from users.
AI agents have been under development for years by various companies. In 2018, Google demonstrated that a computer can do things like call a hair salon to make an appointment for a user or contact a restaurant to book a reservation, all by itself. The staff at these businesses did not know they were talking to a computer. Google, however, did not launch it in part due to fear of public backlash, the news outlet said.
Google is changing its tune now. CEO Sundar Pichai recently said that adding generative AI to search would make it “act more like an agent over time” to “go beyond answers and follow through for users even more” by autonomously executing on their search results.
What this means for your organisation
Agents work where APIs do not.
Many business systems lack clean integrations. Agents that click and type like a person can automate them without new code.
Beyond robotic process automation.
Unlike scripted bots, agents handle unstructured work with little guidance. That widens the tasks firms can automate.
Control and security matter.
An agent that takes over a device can access everything on it. Leaders need rules on permissions, logging and oversight.
Three questions for your next leadership meeting
- Which multi app workflows could an agent run for our staff?
- What permissions should an AI agent have on our devices?
- How would agents change our software and app spending?
The trends and technologies behind this story
From the 311 Starburst, the 311 Strategic Intelligence Unit’s map of the technologies and trends shaping the next 50 years.
Analysis by the 311 Strategic Intelligence Unit, from Matthew Griffin’s article and 311 data. First published on Fanatical Futurist.
FAQ
Why does this matter?
OpenAI might very well be trying to create a new kind of device OS that uses agents to automate tasks and perform actions, and which could replace the need for apps.
About the author
Matthew Griffin Founder, President and Futurist, 311 Institute
Matthew Griffin is a multi-award winning Futurist and expert in Disruption and Innovation, Geopolitics, Leadership, and Technology, who NASA have described as a "walking encyclopaedia of the future" and a "futurist Polymath."
Read full bio
Matthew Griffin is a multi-award winning Futurist and expert in Disruption and Innovation, Geopolitics, Leadership, and Technology, who NASA have described as a "walking encyclopaedia of the future" and a "futurist Polymath." 17-time best selling author of the "Codex of the Future" series, Matthew is the Founder, President and Futurist of the 311 Institute, a global Futures and Deep Futures advisory firm working with royal households, world leaders, G7, G20, and G77 governments, NGOs, and multi-national mid and mega cap firms to help them explore, shape, and lead the next 50 years of business and society.
An award-winning YouTube creator with over a million followers, with an unrivalled global reach and impact, Matthew is a highly sought-after international keynote speaker, lecturer, and mentor who collaborates with global leaders through the United Nations Alliance of Civilizations (UNAOC) and United Nations General Assembly (UNGA) to shape pivotal initiatives such as the UN’s AI for Humanity programme, the United Nations Conference of the Parties (UN COP), and the World Economic Forum in Davos.
As the former Global Head of Cloud, National Security, and Enterprise Sales for companies including Atos, Dell-EMC, and IBM, Matthew has a proven track record of building multi-billion dollar business units and turning failing divisions into market leaders. His ability to identify, analyse, and communicate the implications of hundreds of emerging technologies and trends is unparalleled, and his insights are trusted by many of the world’s most respected organisations, including ABB, Accenture, Adidas, AON, ARM, BCG, Centrica, Citi, Coca-Cola, Dentons, Deloitte, Dow Jones, EY, Google, KPMG, Lego, Legal & General, LinkedIn, Microsoft, PepsiCo, Qualcomm, RWE, Samsung, Siemens AG and Siemens Energy, T-Mobile, UBS, VISA, Walmart, Workday, Worldpay and many others.
Regularly featured in the global media including the AP, BBC, Bloomberg, CNBC, Discovery, Forbes, Khaleej Times, Telegraph, TIME, ViacomCBS, WIRED, and the WSJ, Matthews mission is to help organisations create a fair and sustainable future whose benefits are shared by everyone irrespective of their ability, background, or circumstances.
What future do you need to see?
Choose one to get started on AI and intelligence and the future of your organisation.