Get email delivery of the Cadence blog featured here
Is it possible – or desirable – to take the “art” out of functional verification and let it be a systematic, quantifiable “science?” Panelists at the recent DVCon 2015 conference wanted a systematic flow, but they also said that creativity is still needed to solve new problems.
The panel was moderated by Brian Bailey, technology editor at Semiconductor Engineering. Panelists were as follows, as shown left to right in the photos below:
“It seems that verification is an art rather than a science,” Bailey said. He cited three reasons – the problem is too large, nobody really knows how to automate verification, and if it was a science all the jobs would have gone to India or China.
But times have changed, he noted. Many designs today are built from pre-verified blocks or platforms. Companies are looking for turnaround times of weeks if not days. “To make that a reality we have to rethink verification. We definitely need a lot more automation than we have today. We need to start looking at how to make it more of a science than an art,” Bailey said.
Questions and Answers
Q: What progress has been made to reduce the amount of art in verification, and what else can be done?
Gray – There has actually been quite a lot of progress in the last 10-15 years – we have constrained-random stimulus, functional verification, and verification plans. But is it possible to have too much of a good thing? As you make the car less dangerous to drive, you may have more accidents because people aren’t paying attention.
Knowlson – We want to use production firmware and drivers for validation purposes, but we can’t afford to write verification firmware that rivals production for complex systems. I think we have some work to do to meld these two communities [firmware and software] and get them talking the same language, so we can deliver something that works for everyone.
Murphy – There is definitely an increased awareness of static and formal verification. This has become a very necessary component, especially for SoC-level verification. Progress needs to be made in using static and formal verification almost as a co-equal component with simulation.
Foster – In the past three years, according to a study I recently completed, formal apps grew by 62%. That’s faster than anything else in EDA and it has worked quite well. What is not working well is the whole system aspect. If we think about IoT [Internet of Things] edge-node computing, we have got to examine it from the system perspective. The classic coverage metrics we have today do not work well in that domain.
Bergeron – We are finally at a point where we can draw a line between systems and subsystems and get to some definitions. A subsystem fits into RTL simulation. However, you cannot verify an entire system in simple RTL. You need a mix of virtual platforms and emulation and simulation. I think we’ve gotten the subsystem part fairly well – the system is becoming the problem.
Q: At what point can you start to trust the IP and not have to re-verify it?
Foster – It is critical to deliver high-quality IP, but you still must model the interactions.
Gray – If you want to verify in a certain way, you should separate out the parts so it is easy to take a part and verify it completely. If you don’t imagine it up front, you’ll mix it all together, and verification will be difficult. The person who built it [IP] wasn’t concerned about verification – their concern was PPA and getting RTL complete. They did not take the time up front to imagine how you could verify it.
Murphy – Whoever is building IP is doing so because they believe it’s going to have the best functionality and PPA. Companies that build smartphones are tinkering with power management at every level to save picowatts. They’re not going to stop doing that just because it simplifies verification.
Knowlson – System integration is a hard thing. Yes, the IPs are validated, but bringing them together is difficult. IP teams are between a rock and a hard place because they have to support a diversity of products.
Q: I would argue that verification will remain an art for a while. It’s still very hard for us to say what a test or a regression does. There’s nothing to tell you that you’ve really tested this enough. You need to take a pretty big leap – which is the art part of it – and say you’re ready to take this risk.
Murphy – I don’t deny that the process of getting to completion involves a lot of creativity and judgment, but when you get to that point shouldn’t the definition of where you got to be scientific?
Foster—I do not like the term “art.” I feel that it is “ad hoc” versus “systematic,” and ad hoc is not necessarily a bad thing. I’ve seen teams that can accomplish some amazing things.
Gray – If you’re doing something you already know how to do, it’s predictable and you understand what’s going to happen. If you add something you don’t know how to do, at that point you have probably added the artistic element back in, because you are building a system that hasn’t existed before.
Q: Art is defined as a solution achieved through creativity. Science is a repeatable recipe. Shouldn’t the panel be titled art and science? You cannot escape creativity even if you have a recipe.
Murphy – If the creative process is what defines how far you get in verification, then you have a problem. You need a quantified process. Within the bounds of that quantified process, you can innovate and be creative.
Knowlson – If it’s complicated, we can use best-known methods and we know how to do it. If it’s complex, we don’t really know how to do it. This is where art falls in.
Q: We talk about “shift left” and bringing software into the verification process. Why can’t we shift right and use what the product team or the verification team has created? It would be really nice if we could hand an API directly to the software team.
Knowlson – We’re actually doing that with new IP. It doesn’t work so well with legacy.
Q: Is it possible for verification people to create beautiful code if they use SystemVerilog?
Gray – Even with the constraints placed upon a language, if you know what the rules are and you know what you’re doing, you can be creative.
Parting comment (audience) – Art has everything from Michaelangelo to Jackson Pollock. If you reproduce art without being creative yourself, it’s called forgery. So you need to understand what you’re doing and apply your own creativity to the problem. We need to continue to focus on problems in new ways. If we keep that in mind, the future will be pretty good.
Other DVCon 2015 Blog Coverage
DVCon Accellera Panel – What’s the Key to IC Design Efficiency?