EDA on AWS Graviton

At the Arm DevSummit, there were several presentations on the first day about EDA on Graviton. Graviton is an Arm-architecture chip developed by AWS (in its Annapurna Labs group). There was an original version, now known as Graviton 1, a couple of years ago. I've not heard anyone say it explicitly, but I think that this was a proof of concept. It was not in a truly leading-edge process node. The real plan was for Graviton 2, which has higher performance than x86. Meanwhile, all the software porting could be done on Graviton 1, and it could be used to seed the AWS market for Arm-based instances. 

I wrote about Graviton briefly in my post HOT CHIPS: The AWS Nitro Project. I wrote about using Graviton 1 for EDA in EDA in the Cloud: Astera Labs, AWS, Arm, and Cadence Report.

I wrote about Graviton 2 in my post Xcelium Is 50% Faster on AWS's New Arm Server Chip. And I covered how AWS/Annapurna migrated from on-premises data centers to first AWS and then Graviton 2 in my post Climbing Annapurna to the Clouds.

Inside the Cloud

One session after the opening keynotes was called Inside the Cloud. This was a discussion between:

• Chris Bergey, SVP of Infrastructure line of business at Arm (top left), moderator
• Don MacAskill, CEO of SmugMug and Flickr (top right)
• Liz Fong-Jones, Principal Developer at Honeycomb IO (lower right)
• David Brown, VP of Amazon EC2 at Amazon AWS (lower left)

Dave Brown started off giving a bit of the history:

AWS started in 2012 building an Arm offload processor, and that grew into Project Nitro [see above for a link to my post that goes into that in some details]. They then decided to provide a server chip and so we launched Graviton 1 in 2018 (although we just called it Graviton then). At the end of last year, we launched Graviton 2. From a price-performance point of view, it is better than other instance types.

Chris: So Don, you gave Graviton and Graviton 2 a whirl?

Don: We started as soon as we heard AWS would bring Graviton to their offering a couple of years ago. We figured we could deliver massive power at lower cost. We were paying for superpremium compute that we weren't actually maximizing. We expected lower cost but we thought we would have to sacrifice performance to get value, but actually there's no performance sacrifice.

Dave: The benchmarks we ran showed about 40% better price performance, Java interpreted seeing over 20%, and so on. That's for Graviton 2.

Liz: At Honeycomb, we started using Graviton 2 earlier in the year and just started pushing it out to customers. We are seeing 30% improvement in performance and each machine is 20% lower cost. Prototyping took about two to three weeks. Our applications are written in Go so it just compiled.

Chris: How was the experience making the transition? Painful?

Don: It took about two weeks on the first Graviton to get a large compute load up and running. Most packages and ecosystem was already compiled and just worked. It took a couple of weeks to work through the dependencies and test. And then we launched. When Graviton 2 came along, that was a one-day switch.

Dave: I remember there was a tweet the day after we launched Graviton 2 that said the entire production system was now running on Graviton 2.

Chris: What about the software ecosystem?

Dave: It is going faster than we expected. People expected it to be harder than it was. We put Graviton 1 out there to show Arm is in the cloud. And we saw lots of the container and open-source package all move to support Arm. By the time we put out Graviton 2 at end of last year, the ecosystem was there. My advice to customers is “take one or two engineers, and just see what happens.” It's easier than anyone expects.

Don: It's not often a technology comes along and you can just cut out 40% of your cost without needing to negotiate a contract. Just compile.

Chris: I recently had a call with the CEO of a large silicon valley company, and when I introduced him to Graviton, he said it was the best meeting he'd had all week. So Don, what's your view of the future of the Arm architecture?

Don: We were a very early AWS customer, using it in production before it was even announced in 2006. One amazing thing that cloud computing has enabled highly customized instance types so we can fine-tune our CPU to memory ratio, whether we need a GPU, and so on. So I think one of the things I’m super excited about is that AWS has continued to roll out new instance classes based on Graviton 2 so you can now do that in the Graviton world, not just the x86 world.

Chris: Dave, did I hear right that there are some instance types where people can try Graviton for free?

Dave: For the lowest cost instance type, we launched a trial period until end of this year, you can get 750 hours per month free on T4G instance type (Graviton 2).

Chirs: Liz, what do you see as longer term impact?

Liz: The thing I’m excited about is its better price (and watts) per compute. So not just saving money but also to do better for the environment

Chris: Thank you, everyone.

Scalable Cloud-Based Simulation and Characterization

Later in the day, there was a session with Cadence and Arm on, basically, Xcelium and Liberate Trio in the cloud. The presenters were:

• Brandon Bautz, Cadence
• Frank Shirrmeister, Cadence
• Bhumik Patel, Arm

A lot of the presentation was on the capabilities of Xcelium and Liberate. I've written about those a lot before, so I'll focus only on the cloud aspects.

Bhumik detailed the newish instances of Graviton 2-based instances that have more memory than the original Graviton 1-based A1 instance. This is ideal for EDA. One thing that Arm themselves found was that there wasn't enough memory to always be practical before. As Arm's Tim Thornton (director of Arm-based engineering) said:

With the introduction of the first AWS Graviton processor last year, we proved our workflows could run on A1 instances very well. However, the available memory of the largest A1 instances was too small for other workloads. The launch of the AWS Graviton2 and Amazon EC2 m6g instances changes the landscape considerably. The R6G instance with up to 512GB RAM will allow us to address even more workflows.

He went on to look at silicon workflows and pointed out that the cloud is becoming the new signoff platform. It has huge compute requirements but it is only needed right at the end of the project so it makes no sense to provision for in-house.

For library characterization on A1 (Graviton 1) Arm found that it was slower than x86. But the A1 cost is 3.8X cheaper than an X86 instance, resulting overall in a 35% cost reduction for the same throughput (more instances but much cheaper). Graviton 2 is faster than the equivalent x86 instance, and cheaper (but presumably not 3.8X cheaper). As Arm's Philip Moyer said, their turnaround time was reduced from a few months to a few weeks.

Chip Design on Arm in the Cloud

Later, Tim Thornton turned up in his own presentation, along with AWS's David Pellerin, head of BizDev for Semiconductor at AWS. Some of what they said duplicates things I've already discussed in this post, so I'll just hit a few highlights.

He talked about the cooperation between Arm and Cadence, focused on simulation and characterization since they use the most cycles (the topics that Frank and Brandon talked about above). Breakfast Bytes even got a mention!

Dave compared Graviton to Graviton 2 (see the above image). As you can see, Graviton 2 (compared to Graviton 1) has:

• 7X the performance, 4X the compute cores, 5X faster memory
• Built in 7nm (compared to 16nm for Graviton 1)
• Up to 64 vCPUs, 25 Gbps enhanced networking, 18 GBps EBS bandwidth (all much bigger than Graviton 1, of course).

A case study was Mediatek. They needed to verify a complex, 5G cell phone chip in time to meet customer schedules, but did not have adequate capacity of high-memory servers in their on-premises data center environment in Taiwan. So they migrated static timing analysis EDA workloads to AWS, using 12M core-hours of computing over a two-month period, using US West 2 region and up to 1000 R5.24xlarge instances. The met their aggressive tapeout schedule to release the world’s first 5G integrated SoC.

Tim's summary slide of EDA performance on Graviton 2:

Conclusion

If you are using AWS to run EDA in the cloud, then you should be using (or at least considering) Graviton 2 instances. They are faster than x86, and lower cost. The instance names to look for are M6G, C6G, and R6G.

Sign up for Sunday Brunch, the weekly Breakfast Bytes email.