Toyota’s new Generative AI tool brings designers concept drawings to life

A few years ago I shared some news that General Motors was starting to use Generative AI to help design its cars and make them more fuel efficient by removing weight from them. Now, as GenAI makes more inroads into helping design cars Toyota Research Institute (TRI) has unveiled a new innovative tool that lets vehicle designers generate new car designs and sketches using just text prompts. Furthermore, it’s intelligent enough to let them incorporate precise stylistic attributes such as “sleek,” “SUV-like” and “modern.” The designers can also optimize quantitative performance metrics to create an initial prototype sketch.

The company said this innovation will empower designers to explore their creativity while ensuring efficient and effective design development.

TRI researchers have also published two papers describing how the developed technique can be incorporated into other text-to-image-based generative AI models. These papers shed light on the tool’s image-generation process.

Watch it in action

The team merged principles from optimization theory, which is extensively used in computer-aided engineering, with text-to-image-based generative AI. As a result, the algorithm allows designers to optimize engineering constraints while preserving their text-based stylistic prompts for the generative AI process.

Designers can now implicitly integrate vehicle constraints such as drag, which directly impacts fuel efficiency, and chassis dimensions like ride height and cabin dimensions, which affect handling, ergonomics, and safety to enhance their image generation.

“Current Text-to-Image generative AI tools primarily focus on adhering to the designer’s text-based stylistic guidelines when generating potential images,” Avinash Balachandran, director of the Human Interactive Driving (HID) division at Toyota Research Institute, told reporters. “Our technique allows users to explicitly incorporate and optimize over - engineering constraints like drag or ride height while generating images that adhere to the designer’s stylistic guidelines.”

Balachandran said that such techniques could speed up the creation of new designs by balancing the trade offs between aesthetics and engineering more quickly and efficiently.

“Any designer can use generative AI tools for inspiration, but these tools cannot handle the complex engineering and safety considerations that go into actual car design,” he added. “To build safe and reliable vehicles, our designs must meet engineering requirements. Adding constraints to generative AI essentially allows the user to add guide rails to the generative designs from AI.”

Balachandran said that the project began approximately a year and a half ago, driven by the advancements in Text-to-Image generative AI tools that allowed users to input a prompt and in response generate an image that aligns with the provided stylistic guidance.

“Our vehicle designers in Toyota told us how one of the challenging parts of the design process for them was to come up with inspiration for new designs,” he explained. “They also told us that the back-and-forth iteration process between design and engineering to produce a design that is not just aesthetically pleasing but also has the desired engineering performance and safety measures were hard.”

According to Balachandran, designers and engineers typically come from diverse backgrounds and have different modes of thinking. Consequently, when a designer creates a design, it usually fails to meet the initial engineering requirements, resulting in substantial collaboration with the engineering team to arrive at an optimal solution.

This iterative process, coupled with the inherent tension between design and engineering, contributes to the extended duration of design.

“The inspiration for this technique and these tools was not just to spur creativity but also to shorten that iteration loop between engineering and design,” said Balachandran.

Toyota stated that during ideation sessions with designers, one idea that resonated with them was the concept of an “AI assistant” that proposes new designs by leveraging multiple diverse data streams. This sparked the idea of integrating generative AI into a tool incorporating diverse data streams, including engineering constraints, to generate innovative designs.

“By integrating generative AI technology, we found that designers were able to focus on identifying constraints and important stylistic aspects of the design with the assurance that the engineering constraints are met,” added Charlene Wu, senior director of the Human-Centered AI (HCAI) division at Toyota Research Institute. “We believe that our tool will allow them to focus more time on the part of the design process that they enjoy the most and where they can add the most value.”

The company announced that while the technology is currently in the research phase, they are collaborating with teams within Toyota to integrate this tool into their vehicle design and development process. TRI stated that they will continue researching to enhance the quality of life for individuals and society.

“The hope is that by using this tool, vehicle designers worldwide can expand the power of design ideas while at the same time drastically improving the speed of design development,” said Balachandran.

“Generative AI is a powerful new tool, and across our many research areas, we’re exploring how to leverage it responsibly so it can amplify people.”