Step into the holodeck, Theoriz unveil their mixed room reality concept

If you had to list the most mind blowing tech demos in recent memory, Microsoft's Hololens Augmented Reality headset would need to be included, as would its projector enhanced Microsoft companion Illumiroom. Now though a company called Théoriz from Lyon, France has married both of those concepts to create what they call a "mixed reality room," or “first generation holodeck,” that uses projectors, motion tracking and augmented reality all together. And in their latest technology demo video they managed to make it seem like we're closer to Star Trek's Holodeck than ever before. Théoriz’s mission is as much artistic as tech oriented, so the engineers are both bohemian and code savvy.

"We are a team mostly composed of creative engineers," says Théoriz co-founder David-Alexandre Chanel. "Engineers who have an artistic sensibility and also do good code."

To wit, the company has created some very technical and very whimsical projects, including an art installation called "Doors" featuring portals that open up to an infinite space and change perspective as the viewer moves, and "Are You my Friend," an industrial robot that communicates with the exhibit goers via a keyboard.

Take a look around 

Art aside, the mixed reality room tech, as you can see, is impressive. The team tracks the camera, typically a red model that can record and output in real time, with an HTC Vive Tracker, and feeds the data to a computer running the Unity game engine.

That generates digital environments like flying space skulls, a Minecraft like room with holes that open up on the floor and geometric shapes that interact with actors to form stairs, wells or small hills.

The computer then syncs everything together, so that when the camera operator pans or tilts, the Unity scenes tilt or pan to match. Those are then beamed into the room via six projectors, four for the floors, and two on the walls. At the same time, three Kinect style 3D cameras, combined with Théoriz's in-house "Augmenta" system, detect the position of the actors so they can interact with the environment. Everything must be processed and played back in real time by the Unity based system, something that required some clever coding and computing horsepower.

In the resulting videos, live actors interact seamlessly with virtual environments, creating a hallucinogenic effect.

"It's called mixed reality because we use and merge things from the virtual world with reality," says Chanel.

For instance, dancers can make the walls "move" with their movements and bat away flying asteroids. In the latest demo video, above, actors interact with bizarre geometric environments, opening up holes in the floor where they move and walking up fake stairs.

Though most of the tech is off the shelf, none of it is intended for consumers, at least, not yet. For now, the company wants to just sell its services for things like music videos, dance performances, art installations and other live events. At the same time, they're improving the tech to make it more realistic and immersive.

"We think that by changing the content creation process, we can open new creative possibilities and achieve unprecedented kind[s] of visuals," says Chanel.

The next project will test everything Théoriz has learned so far, both artistically and technically.

"We're trying for the first time to show an artistic video with two dancers," Chanel says. "And they're going to dance and interact in the virtual world, moving through different kinds of totally surreal scenes."

Eventually, Théoriz wants to make its software available to other companies, but for now it's just trying to make its services more compelling for artists and audiences. "It's a new field," says Chanel. "We still have to evangelize it and create demand so it can eventually find its place." And the best way to do that? "Seduce the audience with something new, poetic and unexpected," he says.

Source: Théoriz