One question that prompted a lively discussion at the recent Cadence Mixed-Signal Design Summit was whether design engineers should do their own verification. This is a particularly good question for analog and mixed-signal design, where the tradition of separate verification teams is not as strong as in the digital world. At the summit, participants in a Q&A panel made a strong case for separate verification teams in mixed-signal design.
I wrote previously about user presentations at the summit, and the five users I wrote about were members of the panel, which primarily discussed mixed-signal SoC verification. Here is what they had to say when asked, “does it make sense for designers to do their own verification, or does it make sense to have a separate verification team?”
“You definitely want a different person running your verification. It doesn’t matter whether it’s mixed-signal or digital. You want separate pairs of eyes verifying your design. You will get much better coverage that way than letting the designer verify his own IP, because he will verify it for zero bugs.”
--Yuval Shay, staff engineer for mixed-signal verification at STMicroelectronics.
“Obviously there has to be a team with a dedicated focus. You have people dedicated to overall verification for the SoC.”
-- Kumar Abhishek, senior analog and mixed-signal design engineer at Freescale Semiconductor.
“I’m approaching your design as if everything is wrong. My main goal is to break the design. That is what differentiates me from a designer. You don’t try to break your own design.”
– Prasanth Aprameyan, senior verification manager at Micron Technology.
“Developers have a bias. That’s why you need a second set of eyes looking at it.”
–Robert Milkovits, director of technical support at Jazz Semiconductor.
“For us, the most efficient way to proceed is to have a dedicated verification team. We find bugs, and then it’s more efficient to hand it off to the designer and say ‘why isn’t this working?’”
– Jess Chen, senior staff engineer at Qualcomm.
Later on in the panel, Aprameyan noted that Micron separates functional verification teams from performance verification teams. Milkovits talked about the need for a “joint venture” between designers and verification engineers. There was also considerable discussion about who should write analog behavioral models for simulation. “Analog designers usually don’t want to get involved, but they’re the ones who have the knowledge,” Chen noted.
My perspective? On the analog side, I think there could be a real benefit in having a separate verification person or team. Analog designers are coping with increased functional complexity, higher performance demands, and low-power requirements. Having someone who can do behavioral modeling, run detailed regression tests, and treat verification as more than a late-in-the-day afterthought can help meet these demands.
On the digital side, where separate verification teams are commonplace, it’s more a question of who does what in terms of verification. If the designer can do some early checking before turning his or her block over to a dedicated verification team, it will help reduce the overall verification burden. Static checking with formal tools, such as Cadence Encounter Conformal Low Power or Cadence Incisive Formal Verifier, can help with some of that early block verification.
It’s kind of like the publishing world. If you write an article for a newspaper or magazine, someone will (hopefully) edit it, and it’s very helpful to have that extra set of eyes. But with copy editors in short supply, writers should carefully read their own copy and run static verification tools that check spelling and grammar. In any area of endeavor, the more checking you can do up front, the better.
we have a different Verification team, challange is that Verification team don't find their job interesting & there is not growth path for them
Definitely a separate verification team, which can enjoy a degree of separation and devote its objectives to the system requirements. Requirements are the driving force behind new verification methodologies that stress starting the process at a higher level, e.g., VMM, OVM, in combination with ESL and mixed-level (abstraction) design and simulation.
Janick Bergeron's Veriification Guild website is a great place to read and learn about verification and the philosophy behind it. Even if you're not into VMM (e.g., part of the OVM camp), there will be a lot of useful ideas for you. [n.b., Janick is now with Synopsys as a result of his start-up's purchase (Qualis) but a neutral Verification Guild was part of his deal!]
I think it is not a question of either/or, but a healthy balance of both. No Engineer should entirely rely on others to debug their work. This type of situation fosters sloppyness in the designer and resentment and distrust by the verifiers. However, due to the multitude of cognitive errors exhibited in human nature that none of us are entirely free from - such as wishful thinking, confirmation bias, cognitive dissonance, tunnel vision, etc , having independent eyes doing secondary verification goes a very long way to addressing these problems. While most designers are fully aware of the need to try to break their designs, to quote Taleb's 'Black Swan' book, it is the things 'out of left field' that designers are not so good at testing for - things that people unfamiliar with their project, (i.e. customers) might do to it. From a practical side, with tight time-schedules, verification is definitely something that can be developed in parallel by a separate team.
This is a nice article and I fully agree. The problems are probably manpower, time and money to really split analog/MS design into pure design and verification. Often there are only very few specialists available and in analog and RF design and verification are not so easy to separate - if there is not much time.
One important aspect to some degree often overlooked is that having a full verification flow from system planing to block design to final verifications would highly smooth all activities, enabling better communications on interfaces and leading to early and efficient verification. Designers tend to be conservative, so great tools such as the AMS Designer are in so common use than they should, and also really advanced model libs are often not available. So people stick to Matlab, pure analog simulators, and simple spice-like subcircuit models and are this way simply not on the state-of-the-art wrt to design & verification flow. Not to mention the analog vs digital boarder...
All well-known problems but hard to earn money on it.