Developing an AI Inference Chip with Cadence Flow in Azure Cloud

At last year's CadenceLIVE Americas, there was a presentation by d-Matrix about designing its AI inference chip in the Microsoft Azure Cloud. You've probably not heard of d-Matrix (in fact when I saw the name, I mixed them up with dSpace, a German company that provides code-generation and test tools in the automotive industry). d-Matrix describes themselves:

d-Matrix is building one of the most disruptive computing platforms for AI in cloud and edge data-centers. A pioneer of in-memory computing (IMC) in the data-center, d-Matrix has attacked the physics of memory-compute integration using innovative circuit techniques, and built a 40-100 TOPs/W datacenter inference engine, with a path to further 1000x gains in compute efficiency.

The session was presented by Farhad Shakeri, d-Matrix (head of IT Cloud)

Farhad said that the presentation would cover how d-Matrix built IT infrastructure entirely in the cloud which was used to develop its first silicon (Nighthawk).

When the company was created, it evaluated all options: on-prem, hybrid, and cloud. With an aggressive schedule and money limited by the seed funding, it was necessary to keep the team focused and costs in check. So cloud-only was really the only path forward. On-prem and hybrid were not really feasible due to the high up-front costs and the time required to set everything up. In the cloud, upfront cost is low and setup is quick. Compute and storage were scalable and security was taken care of. So d-Matrix worked with Microsoft Azure and Six Nines IT for cloud setup. And, of course, Cadence, since d-Matrix uses Cadence exclusively for EDA tools.

d-Matrix flow consisted of:

• Digital flow
  • Xcelium for RTL simulation
  • Genus for synthesis
  • Jasper for clock domain crossing
• Analog flow
  • Pegasus for physical verificaiton
  •  Virtuoso for schematic capture and layout
  • Spectre for circuit simulation
• IP
  • Tensilica Xtensa core

The diagram above shows the timeline for setting up the environment. At first they used Azure NetApp but it was too cost-prohibitive (they may reconsider it later). Instead, they built their own VM server with a lot of disk, using NFS and AutoFS on the client side to mount file systems. They used encrypted backups to the Azure blob for disaster recovery. The only change they did late on for tapeout was to expand their file capacity.

The peak usage of cloud was during tapeout, as you can see in the above diagram. Storage was 26TB of disk, a mix of premium SSDs and standard hard drives, with snapshots and nightly backups for disaster recover.

Silicon came back in February (2021). It has been a success. We could demonstrate working silicon within 2 months of receiving the chip. The photo shows the silicon, evaluation board, and demo station. I hope this success will encourage other startups to adopt a similar cloud-based model.

Lessons Learned

Why Azure Cloud Works:

• All in one place
• Scalability and availability of compute
• Execute to aggressive schedule (14 months to tapeout)
• Independent and parallel compute for verification and layout
• Broad ecosystem of partners

Lessons Learned

• Don't be afraid of the cloud
• With planning and automation you can keep costs down
• Disk space grows forever and can get out of hand
  
  • Take advantage of the Azure blob archive
  • A few minor scripts to encrypt the data before sending to the blob
• EDA + Cloud still has some challenges, work in progress

Watch the Video

You can watch all the CadenceLIVE videos on the topic of cloud (including this one) starting from the Cloud Page.

Sign up for Sunday Brunch, the weekly Breakfast Bytes email