A standard is quietly emerging to help verification engineers deal with coverage information from different sources - and if that's a concern for you, there's still time to get involved. The Accellera Unified Coverage Interoperability Standard (UCIS) committee is developing a draft standard for an API that will allow engineers to merge coverage data from different verification tools into a single database. This includes tools from different vendors as well as different types of tools (specifically, simulation and formal verification).
The UCIS committee was launched in November 2006 with several initial goals:
The problem is simple. As of today, every verification tool has its own way of tracking coverage, and every vendor has their own way of accessing and using that coverage information, which only works for tools coming from the particular vendor. Yet most verification environments include tools from different vendors, as well as different types of tools.
If you're using the Cadence Incisive Enterprise Manager for metric-driven verification and planning, for instance, integrating coverage information from a third-party simulator or internal verification tool is difficult without using proprietary APIs and libraries that can interpret the results. If you're using a formal verification tool, it may be difficult to compare formal coverage to simulation coverage and determine how far you can get with formal.
"What we are trying to standardize here," said Richard Ho, engineer at D.E. Shaw Research and UCIS committee chair, "is a method to bring in metrics from different verification processes - simulation, formal verification, and maybe emulation. We want to bring it into a single place so designers can get a single view of what they've verified and what remains to be verified."
The UCIS API will allow multiple tools from multiple providers to write to a UCIS-complaint unified coverage database. (Source: Accellera)
At the moment, a draft API standard is under circulation in the UCIS committee. While it currently focuses on simulation coverage, the intent is to add the necessary information for formal verification, Richard noted. He said that the committee's goal is to have a balloted standard by the end of 2010.
While the draft API standard was based on an original donation from Mentor Graphics, Cadence representatives helped improve the donation by making it more robust and extensible, said John Brennan, Cadence Incisive Enterprise Manager product marketing director. Now, he said, "we want to get the word out to the users. We want them to understand what it is, how it's going to help, and why it's good for the industry." John added that "the challenges UCIS is looking to solve for the industry are things that companies like Cadence have already solved in their own way. We expect to be able to leverage that expertise to help improve verification productivity for all users."
It is important to note what the draft API standard is not. It does not establish a single coverage metric for simulation and formal verification. It does not mandate any particular type of metric, or interpret coverage metrics. It does not do away with proprietary vendor databases. It is, as Richard said, a "first level data interchange" that provides an API so information can be gathered and viewed together in one place. It also includes an implicit data model, and is expected to include an XML interchange format in the future.
Still, this "first level interchange" will be very helpful for any verification team using tools from different vendors, or data from engineers in different geographical locations. It will allow users to compare coverage information from simulation and formal verification, and check off a block as "done" regardless of the methods used.
Companies involved in the UCIS effort include Cadence, Mentor, Synopsys, Springsoft, IBM, Jasper, Sun, D.E. Shaw Research, and Freescale. "It's a really critical time right now," Richard said. "The draft is not all sewn up yet, and if you have any interest at all in this kind of coverage, check it out. We still have room for people who can help correct things. People who are enthusiastic are always welcome." Further information is available at the UCIS web site.
You missed out a key participant from the list of participants in UCIS, somewhat surprisingly. Paradigm-Works is heavily invested in this effort from the very beginning and we bring a much needed perspective to this effort based on our experience at multiple end-users at the ground level.
I encourge the entire verification community, vendor/end-users alike, to participate and contribute at this formative stage of the standard.
-Dr. Ambar Sarkar
Chief Verification Technologist, Paradigm Works
Holly -- the questions you raise about simulation vs. formal metrics are good ones. These questions outline a next-level challenge for UCIS. The purpose of this short blog is to provide a quick update on what's been happening in the committee. I expect to revisit the topic of coverage metrics in the future.
This is Holly from Jasper Design, your long time fan. I want you to know that Jasper is very much a proponent of UCIS, in fact, our CTO Rajeev was one of the original organizers. However, I do think you “buried the lead” in this article, the very interesting technical challenge of “what is inside the blue disk?!” It is a very pretty “marketing vision”, that simple blue cylinder...but really hides the complexity of the metrics issues for different verification tools.
Metrics for simulation are inherently so different from metrics for formal. How are they defined, how interpreted, can they ever really be shared or rationalized? Not just put in the same database...this is the next and real technical challenge; so blithely alluded to, in passing, almost at the end of the piece.
I hope that you accept the challenge of taking this analysis a step further, it is an issue of real importance in verification! The Dec 2009 ITRS roadmap predicts increasing use of formal in coming years, to address chip complexity, exacerbating the challenge...