This is my third and final blog
entry in a series using quotes from famous Hollywood movies to highlight the
key concepts about metric-driven verification (MDV). As a reminder, here's the four-phase
Cadence MDV flow:
In my first post I dealt with the "plan"
phase and the concept of planning for both coverage and check metrics from the
very beginning of a project. In my second post I discussed the "construct"
phase, in which the verification environment is built and the detailed metrics
are defined. This same post also covered the "run" phase, during which the
metrics are actually generated by multiple engines. Finally, I come to the "measure and analyze" phase, which includes determining whether or not verification is
complete enough for front-end signoff.
The "run" phase generates lots of
metric data, typically many gigabytes, and so the final phase must provide a
way to compile, compact, abstract and analyze the data efficiently. For most of
the project duration, verification engineers hope that a weakness (failing
check) is found during this analysis, since this indicates a bug in the design
or verification environment. It is common for the verification manager to
generate a "bug rate" chart showing the number of bugs found per week on the
project thus far. With an MDV flow, such a chart will usually show a rapid ramp-up
in bug discovery during the early portion of the project, a busy peak period,
and then a slow tail-off as there are fewer and fewer bugs being discovered.
Reaching the point at which new bugs
are being found at a very slow rate (say, one per week) is typically one of the
signoff criteria for the verification lead. Checks by themselves are not
sufficient, since it is possible that the verification environment is simply
not exercising the portions of the design still containing bugs. Coverage metrics
help to solve this problem by showing areas of the design not being
sufficiently verified, another form of weakness. Ultimately, what the
verification team wants is for all the coverage points to be reached while all
the checks are turned on and reporting no failures.
Of course, limitations in the
verification environment may prevent all the coverage points from being
reached. In this case, the verification engineers might re-examine the verification
plan to see if all the metrics were properly specified. If a particular
coverage point seems impossible to reach in simulation, they might use formal
analysis to see whether it's possible to reach it at all. Simple coding errors
sometimes isolate sections of the design so that the logic is unreachable, and
only formal analysis can detect this.
On the other hand, if formal
analysis can reach the coverage point, its trace can be very helpful in hitting
that same point in simulation. Alternatively, the verification team might be
happy to let formal "take credit" for hitting the coverage point and marking it
as done. For any coverage points not yet resolved, the team goes through
another iteration of the MDV loop, possibly tweaking the verification plan or
the environment or perhaps just running more constrained-random tests.
This concludes my short blog series
on the benefits of MDV as expressed by movie quotes. It's been fun writing
these posts, and I hope that it's been enjoyable and diverting for you to read
them. I had lots of ideas for quotes that I didn't use, but instead of sharing
them I encourage you to post some comments with your suggestions. Let's make
this a dialogue instead of a one-way communication!
truth is out there...sometimes it's in a blog.