Leveraging Jasper UNR App for Code Coverage Signoff

24 May 2022 • 5 minute read

Adding tests to your test suite does not generate any further helpful coverage at a certain point in the verification process. It may be due to redundant code, fewer test cases, or there may be some uncovered RTL code. Such issues may result in a longer duration for verification closure. Jasper UNR App addresses this problem by automating the previously tedious, time-consuming code coverage analysis process. The Cadence® Jasper Coverage Unreachability (UNR) App saves weeks in attaining verification closure and automatically generates a database showing where coverage holes are.

By determining coverage for >99% of the FPGA or SoC, the Jasper UNR App can help you avoid countless hours running around in circles looking for that golden test that will cover the uncoverable. The analysis time can be reduced if the UNR app is run after 50-80% coverage (design-dependent). The figure on left displays the stage where Coverage Unreachability App can be used during the development life cycle by designers, verification engineers, and system integrators.

The blog provides an overview of how the UNR app helped Broadcom achieve code coverage signoff for highly configurable and constantly changing design using Cadence Jasper RTL Coverage Unreachability App in less time and with less effort.

Code Coverage Signoff

Code coverage measures the quality of RTL code execution while simulating the test cases. Coverage has become a key technology in the pursuit of efficient and accurate verification of large designs. Code coverage signoff is achieving 100% code coverage on the design and ensuring we have the proper techniques to sign off the code coverage at the end of the project.

For a highly configurable and constantly changing design, the idea of the code coverage signoff is to ensure that

We can signoff the design in every configuration
We have completed the verification of the system
Design under Test (DUT)
DUT is a highly configurable CPU core that can be configured from a tiny microcontroller to a very complex application-specific or audio processor. Its configuration can be controlled using either defines or different parameters. Defines might control the values of different parameters like cache, depth of the pipeline, or the number of ports available on the I/Os. All these design aspects are configurable on the same set of RTL.

Coverage Closure flow

Generate the tests
Pass them through regression and testing
Merge the coverage for tests to generate the merged coverage database

Verification engineers analyze this database and decide whether to generate new tests or sign off the holes as they are unreachable/ cannot be hit. Traditional code coverage signoff mechanisms such as merging the code coverage from a regression suite and analyzing the reports suffer from issues and can be:

Hard to maintain in a changing design
Very hard to reuse across different revisions of a chip
Not portable across vendors

The other key aspect of this design is that it has a common testbench and verification infrastructure to verify every core configuration. So, the verification infrastructure scales according to the configurations of the design.

Motivation

In this configurable design, several aspects of the code were used only in certain configurations in certain modes and not used in other modes. Also, the number of holes is from constants and from such parts of the design, which are either not going to be used or will be removed during the optimization phase. These issues may result in less time to complete the code coverage analysis before the end of the project.

Solution

Jasper Unreachability (UNR) app was used to generate a list of points which can never be covered either by design as either these are tied off or are never going to be exercised after UNR formal analysis.

The flow to Coverage Unreachability App is as shown below:

Results

Here, Coverage unreachability flow is targeted for uncovered block, expression, toggle, and coverage.

In the simulation, expression can be scored using the sum of products (SOP) scoring, control scoring, event scoring, parity scoring, and vector scoring. This flow targets only SOP and controls scoring modes.

In the block coverage, the simulation generates coverage for both conditional and unconditional statements.

In the case of toggle coverage, the coverage unreachability App support is the same as that of Xcelium.

Some of the results from the entire flow through the Jasper UNR analysis are as shown below. The numbers of toggle coverage holes, block holes, and expression holes are in pairs of three different columns and are for six different design configurations.

The numbers in the red are the number of holes before doing the unreachability analysis and the numbers in the green are the number of holes after doing the unreachability analysis from Jasper.

The best part is each configuration was completed by the tool in less than an hour as mentioned in the last column.

Summary

In collaboration with Cadence, Broadcom developed a code coverage signoff flow using Xcelium simulator’s constant propagation, Integrated Metric Centre’s coverage merge, and Jasper UNR App. The goal was to have signoff criteria based on coverage reports that do not involve writing custom waivers. The merged code coverage database from regressions was handed over to Jasper UNR App to analyze the coverage holes for unreachability using formal techniques. Unreachable lines of code were removed from the list of holes to create a list of real holes. Having the formal analysis approach to eliminate the unreachable code is key to achieving code coverage closure and on-time delivery of a highly configurable design. So, in conclusion about this entire flow and the benefits of unreachability analysis from Jasper, the constant propagation and Jasper unreachability app helped Broadcom

To reduce weeks of effort of code coverage holes review and signoff to days.
Prioritizing the area of stimulus generation for reachable code.
Eliminating the need for filters for unreachable code.

Subscriptions