Thoughts on the DVClub Talk: "Is it Time to Declare Verification War?"

As noted in a prior post, I had the pleasure of attending a DVClub talk given by Brian Bailey entitled "Is it time to declare verification war?". As suggested by the word "war" title, Brian drew many analogies between the legendary text The Art of War by the Chinese general Sun Tzu, and the strategic and tactical challenges of verifying a complex device under test (DUT), complemented by introductions of some interesting emerging technologies.

Brian Bailey discusses verification strategy with the DVClub Silicon Valley audience in terms Sun Tzu would recognize

Reflecting on Brian's remarks, there is one point Brian made that has continued to stick with me: one of Sun Tzu's overarching themes is to take care to separate strategy from tactics, and employ only those tactics that will support the chosen strategy.  In this vein, Brian correctly observed that for too many customers code & functional coverage itself equals "verification".  Furthermore, one of the problems that we [in the verification community] have is that we tend to over-rely on objective feedback (like coverage), and often fail to step back an look at the underlying issues behind the data.

Don't get me wrong: I'm a big fan of coverage and metric driven verification, but you have to concede that Brian has a great point.  As with any task that heavily leverages objective criteria -- like the selection of students for college admissions by their SAT test scores for example -- it's easy to obsess on the numbers themselves and forget to evaluate whether the way you are generating said numbers are honestly serve the desired strategic goals, and/or even serving the *current* strategy and not a prior one.

Thus, while high coverage scores generally correlate with a high quality DUT (like high SATs generally predict success in college), verification is actually about measuring the fidelity of the DUT to the spec, and even "perfect" functional coverage scores are not a fool proof guard against corner case bugs.  Hence, it's critical to use overlapping tactics to complement code & functional coverage -- checkers & assertions, scoreboards, formal analysis techniques, etc.

I'm interested to hear your take-aways from either the Silicon Valley or Boston instances of Brian's talk -- feel free to comment below or contact me offline.

Hope to see you at the next DVClub!

Joe Hupcey III

P.S. Full disclosure: The Art of War has been a favorite of mine since I first encountered it back in my own SAT prep days. I often reference the text myself, and suffice to say the urge to extend it to all sorts of cases is very powerful.  As such, I have to applaud Brian for his restraint for not overdoing the analogies between verification and Sun Tzu's marshal guidance. Bravo!