New MEMS clock accelerates AI training and improves energy consumption

Artificial Intelligence (AI) is changing everything in data centers: New AI-specific chips, new cooling techniques, and new storage drives. Now even the method for keeping time is starting to change, with an announcement from SiTime that the company has developed a new clock that is optimized for AI workloads.

The company says the development will lead to significant energy savings and lower costs for AI training and inference. SiTime was able to achieve these savings by using microelectromechanical systems (MEMS) as the core timekeeping component instead of traditional quartz crystals.

[youtube https://www.youtube.com/watch?v=jdV2icf5Pik?si=kzmMTQH6vnBnPLX0]

The Future of AI, by Keynote Matthew Griffin

Almost every part of a computer has some kind of clock. CPUs, GPUs, network interface cards, switches, and sometimes even active interconnects contain their own timekeeping component. For more traditional computing workloads, these clocks usually fall into two categories: Fast, precisely timed clocks or clocks that are well-synchronized across multiple GPUs (or CPUs), says Ian Cutress, chief analyst at More Than Moore and who works with SiTime.

“The problem with AI is that it’s doing both,” says Cutress. “You want your chip to go as fast as possible, but then you also want to synchronize across 100,000 chips.”

SiTime’s Super-TCXO clock combines the functionality of ultra-stable and well-synchronized clocks into a single device, providing synchronization that is 3 times as good as a comparable quartz-based component at a bandwidth of 800 gigabits per second, in a chip that’s one-fourth the size.

AI is a data-hungry beast. And yet, expensive and power-guzzling GPUs sit idle up to 57 percent of the time waiting for their next batch of data. If data could be served up more quickly, GPUs could be used in smaller amounts and more efficiently.

“You need faster bandwidth. Because you need faster bandwidth, you need better timing,” says Piyush Sevalia, executive vice president of marketing at SiTime.

In addition, one can save a lot of power if GPUs can be put into sleep mode while they’re waiting for more data to load, Cutress says. This, too, requires more precise timing, such that the sleep-wake cycle can happen quickly enough to keep up with the data stream.

For AI, clocks not only need to be more precise, but also synchronized perfectly across many GPUs. Large AI models split their tasks among many GPUs, with each one doing a small chunk of the calculation. Then, their results are stitched back together. If one GPU lags behind the others, the whole calculation will have to wait for that node. In other words, the computation is only as fast as the weakest link. All of the GPUs remain turned on while they wait, so any such delay results in energy losses.

The timing must be precise, well synchronized, and robust - any mechanical vibrations or temperature swings have to be compensated for to ensure they don’t throw off the computation. SiTime’s Super-TCXO aims to combine all three requirements in a single device.

Sevalia says using MEMS oscillator rather than the traditional quartz makes that combination possible. Quartz oscillators use the vibrations of precisely machined quartz crystals - similar to a tuning fork. In contrast, MEMS oscillators are manufactured, not machined, to resonate at a specific frequency. MEMS devices can be smaller, which makes them less sensitive to mechanical strains. They can also be more precise.

“Crystal oscillators have been around since the beginning of time, since compute was a thing,” says Dave Altavilla, president and principal analyst at HotTech Vision & Analysis and who also works with SiTime. “We’ve improved that technology dramatically since its inception. But MEMS takes it another step further beyond what a crystal is capable of. So that’s what I think is being displaced in the market by this new technology is the old way of doing things.”

SiTime’s MEMS-based solutions are already having some success - Nvidia’s Spectrum-X Switch silicon already contains a SiTime device.

Sevalia says he expects the need for MEMS-based timing devices to continue. The company is already planning even higher bandwidth devices, and they’re hoping their innovations will result in even more energy savings.

“We’re just scratching the surface right now in terms of figuring out how much energy efficiency we can bring,” Sevalia says.