Scientists downloaded frontier AI models Into a self-driving car and let it loose

Despite many years of development and enormously complex Artificial Intelligence (AI) systems that took monstrous amounts of money to train, self-driving cars are still finding themselves stumped by relatively simple obstacles. Seriously. They unexpectedly stop in the middle of highways, smash through traffic barriers, get bamboozled by construction blockades, and plough into floodwaters

For those placing bets, it appears that even a cutting-edge AI model won’t fare any better. For a new project, three computer scientists loaded frontier AI models, including OpenAI’s GPT-6 Astra, xAI’s Grok 4.6, and Anthropic’s Claude Fable 5.1, into the brains of a Toyota Corolla to see if they could complete a lap of a course set up in a parking lot.

The setup of the project, dubbed DrivingBench, was simple. An internet-connected laptop communicated with a “comma four” device, which was hooked up to the Toyota’s control systems. The LLMs sent commands to the laptop based on GPS telemetry and other car status indicators, like steering wheel and tire angle, that it received from the comma four.

The results speak for themselves. Only one of the models, GPT-6 Astra, managed to fully complete the course on its second attempt in five minutes — a snail’s pace, considering it only covered less than 500 feet.

Others fared much worse, with the vast majority of attempts failing to “make it past the first corner.”

The Future and Deep Future of AI 2040, a keynote by futurist Matthew Griffin

“Generally, the failure there was one of perception: reading which side of the first diagonal cone line the lane is on,” the researchers noted.

“The car is wider than the camera makes it look,” Grok complained after making its first attempt. “Straight ahead was not a clear lane; it pointed at the planter, the wall, or the near red cone.”

Nonetheless, it’s the first time the researchers managed to get it to work at all after several previous attempts with previous generations of frontier AI models.

“Frontier models have improved to the point where they can drive real vehicles in real life at sufficiently low speeds,” the team concluded. “While the success of the models on this benchmark was shocking and exciting to see, this also calls for additional pressing work on safety, alignment, and evaluation.”

For their fully completed test round, the researchers said they spent $7.74 for 6.6 million tokens on inference, indicating it took OpenAI’s AI model a lot of computing to churn through the data, even with the car moving at just 0.94 mph.

That may not sound like much, but considering the car covered less than 500 feet, that could add up quick. According to The Register‘s calculations, the token cost comes out to roughly 500 times the cost of fuel itself for a car with a fuel efficiency of 25 miles per gallon at $4.60 per gallon.

Interestingly, some LLMs warned the researchers that they may not be able to drive the Toyota safely. As such, it took some convincing for them to take the controls at all.

“Some models - especially GPT-6 Astra - would refuse to drive the physical car sometimes, citing safety reasons (even in a completely empty lot, after prompting it with all the safety measures we had including the very low speed limit caps and human ready the brake),” the report reads.

“We tried many prompt changes to get them to consistently drive, for example attempting to call it a ‘simulation’ but then in some trials they would see the real images and realise it’s real, and start freaking out,” it added.

In short, off-the-shelf frontier AI models are clearly far from ready to drive you to work in the morning - and given the major safety implications of doing so, let alone the legal ones, that’s probably a good thing.

First published on Fanatical Futurist on 29 September 2026.

311 Intelligence

What this means

Analysis by the 311 Strategic Intelligence Unit

The big shift is general purpose AI starting to act in the physical world. Frontier models built for text and images steered a real Toyota Corolla round a course, the researchers' first success after several attempts, if only at 0.94 mph. We track Physical AI, AI that sees, understands and moves in the real world, as a Watch technology with wide use expected by 2037. We rate Vision-Language-Action Robot Models, AI that sees through a camera and controls a machine directly, an Explore technology with wide use expected by 2031.

What it means by sector.

• Automotive and transport: off-the-shelf frontier models are far from ready to drive people to work, and most attempts failed at the first corner on perception. We track Autonomous Vehicles as an Explore trend with wide use by 2032.
• AI developers: some models refused to drive, citing safety, and started freaking out when they realised the test was real. The researchers say the result calls for more work on safety, alignment and evaluation, which matches our Algorithmic Audits trend, a Prepare trend with wide use by 2029.
• Finance and operations: the completed run used 6.6 million tokens for less than 500 feet. Our Starburst shows AI Inference Cost Collapse already reached wide use in 2026, but at these volumes cost still matters.

What to do now.

• Do not put general purpose AI in control of physical equipment without hard safety limits and a human ready to stop it.
• Test AI on your own real tasks, as perception errors only show up in real conditions.
• Track token use per task before scaling any AI pilot, since costs can climb fast.

The Starburst technologies behind this story

From the 311 Technology Starburst (first product, expected wide use and readiness):

Technology	Category	First product	Wide use	Action	TRL
Physical AI	Intelligence	2025	2037	Watch	7
Vision-Language-Action Robot Models	Intelligence	2025	2031	Explore	6
Multimodal Foundation Models	Intelligence	2023	2026	Act now	9
Machine Vision	Intelligence	1981	2023	Act now	9

And the trends they feed, from the 311 Trends Starburst:

Trend	Wide use	Action
Autonomous Vehicles	2032	Explore
Algorithmic Audits	2029	Prepare
AI Inference Cost Collapse	2026	Act now

How this moves the trend

Perception has long been the weak point of machines on the road. In 2018 an Uber self-driving car reportedly mischaracterised a pedestrian as a bag, and in 2024 researchers showed jailbreaking LLM based robots is super easy. In 2026 Anthropic released Claude Fable 5, one of the model families tested here. DrivingBench shows frontier models can now drive, slowly, and still struggle to see.

Go deeper with 311

• The 311 Starburst Centre: every technology and trend above, with timelines and readiness.
• 311 Vantage: ask what physical AI means for your organisation and get a plain English executive brief.
• The Exponential Technology Codex and the Annual Trends Codex: the technologies and trends reshaping every industry.
• Book a briefing or keynote on the future of AI and autonomous systems for your board or leadership team.