We’ve never made much noise about our emulation lab. After all, the multimillion-dollar collection of high-performance processors we use to verify and QA our chip designs is a little bit like a secret weapon. But after learning that the lab is now home to the world’s largest installation of Cadence Verification Computing Platform Systems, we decided to open up this amazing facility for a virtual tour. Hidden in an out-of-the-way spot at NVIDIA corporate headquarters is a warren of specialized machines that work around the clock as engineers test the GPUs and mobile processors of the future.

Hardware emulators re-create a specific computing environment so design engineers can test the design and performance of new processors after a chip has been designed but before it’s manufactured. It’s one thing to design a breakthrough architecture like Fermi, it’s another to make sure it works correctly in the real world. It’s simply not feasible to make physical prototypes of these chips and iterate for each design tweak. Other solutions – such as software simulators – are much too slow. Emulation speeds up the testing process a thousand fold.

Emulators are designed to provide an exact replica of actual hardware. (Software tools, in comparison, simulate or mimic what a particular piece of hardware will do.) When an emulator is plugged in to a PC, it’s exactly like placing a physical chip on the motherboard. From then on, chip designers can test away.

This huge cable comes out of an emulator, delivering the pins of the GPU inside. We connect the cable to a graphics card in a test PC.

Due to the cost and complexity, not every company invests in emulation. But among those that do, we submit that we’re pretty intense about it. The simple reason is that having a world-class emulation lab means we can keep innovating ahead of our competition.

“Today’s GPUs, which are some of the world’s most complex devices, have billions of transistors,” said Narendra Konda, NVIDIA engineering and emulation lab director. “There’s no way around the fact that cutting-edge design tools like hardware emulators are essential for designing, verifying, developing software drivers and integrating software and hardware components of GPUs and mobile processors.”

Since 1995 NVIDIA has invested millions of dollars in emulation. Today, the lab covers a vast, roughly 6,000-foot space secreted away behind locked doors. Step inside and you’re immediately surrounded by racks of equipment. Cables and pipes snake along the floors and along the walls, vents and air conditioning units create a constant whir as they work to keep these gargantuan machines cool. The emulators themselves are sleek, water-cooled beasts, each named after a major river.

“Nile” is an 8 year old emulator in our lab, still going strong.

Near the front is Tigris, a snowflake-shape configuration of sixteen chassis that was built to emulate Fermi. It’s physically the biggest emulator in the lab, but no longer the most powerful. That title goes to Indus, a multimillion-dollar steel-blue piece of hardware a little longer than a minivan.

Three and a half years in the making, Indus was designed to handle Kepler, our next-generation chip architecture and the successor to Fermi.  According to Nimish Modi, senior vice president for the System and Software Realization Group at Cadence, “Indus is the world’s largest installation of Cadence Verification Computing Platform systems, Palladium XP.  It’s great working with a partner like NVIDIA to see how our technologies can work together to advance this industry.”

We worked closely with Cadence on Indus’ design – and although it’s smaller than Tigris, it’s more than twice as powerful. It’s stunning to look at Indus’s mass and complexity and realize that all that power represents one chip.

A space ship? No! It’s the “Tigris” emulator in all its glory.

Filling out the lab are Rhine, Nile and a host of other emulators that might be emulating any number of GPUs designed for uses from mobile to gaming to supercomputing to embedded. If an emulator needs more power, Konda and his team can daisy chain them together in the same way gamers improve their system performance by running multiple graphics cards in SLI. The entire lab has an emulation capacity of 4 billion gates, which are the building blocks of a design.

“Indus” : World’s largest emulator, based on Cadence’s Palladium emulator technology.

“Deploying and managing these complex tools requires a very skilled and committed engineering team,” Konda said. “The great work that the emulation team does keeps this state-of-the-art lab humming along.”

NVIDIA’s Emulation engineering team in front of Indus – They’re one of our secret weapons.

Each emulator connects to a number of PCs which are used for testing and can be accessed remotely. So, for example, an NVIDIA engineer in India can log on, boot up, and start running tests at any time of day or night. Since all graphics processing goes back, in the end, to drawing triangles, the tests start there. Can this new chip draw a triangle? Can it draw a red triangle (not blue, not green)? Testing proceeds until everyone is satisfied that the chip can handle the most complex visual computing tasks and is compatible with all the necessary drivers, systems and so on. At any point, the designers might need to go back to the drawing board and repeat the process again. Once a chip graduates from the emulation lab, it’s sent out to be “fabbed” by our manufacturing partners in Taiwan and from there it’s released into the world.

Today, NVIDIA GPUs are powering supercomputers, in-flight entertainment systems and everything in between. They represent some of the most complex technology on the planet. It gives you a new perspective to stand in the emulation lab and think about the advances in these chips  – the millions and billions in R&D, the years of work – and realize each one starts out right here, trying to draw a red triangle.

  • http://www.facebook.com/mathew.everett Mathew Everett

    Its funny how most of the systems almost look unreal, or like blown up versions of PC cases. One thoing missing is huga Nvidia Logos :)

  • http://twitter.com/djalki Daniel Hall

     Just needs a big fan and sink on the side, with a few hundred 6pin power connectors lol

  • http://www.facebook.com/profile.php?id=100000442324023 Rory Phillips

     Thats chuck norris’s laptop

  • http://www.facebook.com/profile.php?id=583792860 Blueberries Cranberries

    The way it’s meant to be played..

  • http://www.facebook.com/profile.php?id=1323529709 Osvaldo Pulido

    To work here wuld be just amazing!

  • http://www.facebook.com/people/Tananset-Jsk/100000286355300 Tananset Jsk

    that was EPIC!!!  
    and half of engineer-team is indian or egyptian! I heard they are perfect at mathmetic
    anyway. no doubt those people is the a team-with-legenday-skill of all kind

    sry 4 bad-eng :)

  • http://twitter.com/frostbyte123 Chopper Bob.

     Must I be the one to say it, Dare I. Tom, Brian, Amorphous are you there? Will it play Crysis?DrFPS

  • http://www.shaneebenzur.com Shanee Ben-Zur

     Hah, great point Matt. We should look into getting some big NVIDIA logo stickers.

  • http://www.shaneebenzur.com Shanee Ben-Zur

     Chopper, way to carry the torch. The emulator itself doesn’t play games. We use it to help us iterate and design the GPUs of the future…so that you can get new GPUs that do help you play Crysis :)

  • http://www.shaneebenzur.com Shanee Ben-Zur

     Thanks for the comment Tananset – we have a very diverse team and everyone is very talented. We couldn’t ask for better engineers.

  • http://www.shaneebenzur.com Shanee Ben-Zur

    LoL. I don’t know if anyone is going to try to tote any of our emulators around, but if someone could, it would be Chuck.

  • http://www.facebook.com/joytech22 Phillip Pirikahu

     Do I get a prize for saying the very first picture looks exactly like 2 cards in SLI?

  • art mak

     The mother of all PCs!

  • http://www.facebook.com/people/Ashwin-Menon/100001292383880 Ashwin Menon

     As an employee of Cadence who has done some work on Palladium, as well as a guy who has a Fermi card in his PC at home, this was an awesome read. Thanks!

  • http://twitter.com/NightLifeLover Nils

    Interesting I wonder how accurate such a system can simulate actual hardware. What about latency? How is DRAM attached?

  • http://twitter.com/Azarobi Muhammad Ahmad Azfar

    Wow. Impressive and informative. cutting edge stuff i should say. Waiting for the Kepler then :) .

  • http://www.shaneebenzur.com Shanee Ben-Zur

     Thanks Ashwin! Appreciate it :)

  • http://www.shaneebenzur.com Shanee Ben-Zur

     :) it’s pretty cool, we actually are able to string emulators together like cards in SLI…so, how about a gold star?

  • http://profiles.google.com/daniel.r.roberts Daniel Roberts

     The Crysis thing is on ongoing joke, he wasn’t serious. I have noticed that the best labs in the world seem to be made up of people from all over the world. I guess it’s because they have the pick of the best. Keep up the good work guys, every gamer and multimedia user is cheering you on. When I graduate I can only hope to work in such a world class lab.

  • Felipe Fernandez

    Just guessing…is Indus emulating the future Kepler GPU?

  • Constantine Tarasenkov

     :D

  • http://profiles.google.com/carlosap78 Carlos Padrón

    wow, thats a mammut size videocard, but does it run linux ?
    seriously what S.O. you guys use?

  • David Malloy

    All I can say is “Wow!” I had always thought simulation of that scale had to be done in software.

    Just out of curiosity, what do the power requirements look like for systems of that size? I imagine it’s pretty consequential, considering the amount of ductwork in some of those pictures.

  • Joseph Johnson

    Or the fact that GOOD engineers are just plain hard to find now days. And it’s mainly due to the fact that people simply don’t want engineering degrees. You’d be surprised how many companies actively seek out talent beyond the countries which they’re based. There’s a number of modern countries that have a HUGE demand for engineers, that their own colleges and universities can’t supply; so they bring the talent to them.

  • HeadShoulders

    So, uh, when are you going to open source that Indus?

  • https://launchpad.net/~flamingolof Olof

    Interesting. It’s always hard to emulate next generation hardware with the hardware of today. Are these rigs full of CPUs, FPGA based or are you producing a lot of custom ASICs that you use internally? Would be interesting to hear some specs 

  • http://www.facebook.com/profile.php?id=1014755001 Justin Beatty

    can i have 1 to emulate tetris, ultra mega extreme ownage pure epicness edition?