Get email delivery of the Cadence blog featured here
In the former blog about Python and Specman: Specman: Python Is here!, we described the technical information around Specman-Python integration. Since Python provides so many easy to use existing libraries in various fields, it is very tempting to leverage these cool Python apps.
Coverage has always been the center of the verification methodology, however in the last few years it gets even more focus as people develop advanced utilities, usually using Machine Learning aids. Anyhow, any attempt to leverage your coverage usually starts with some analysis of the behavior and trends of some typical tests. Visualizing the data makes it easier to understand, analyze, and communicate. Fortunately, Python has many Visualization libraries.
In this blog, we show an example of how you can use the plotting Python library (matplotlib) to easily display coverage information during a run. In this blog, we use the Specman Coverage API to extract coverage data, and a Python module to display coverage grades interactively during a single run and the way to connect both.
Before we look at the example, if you have read the former blog about Specman and Python and were concerned about the fact that python3 is not supported, we are glad to update that in Specman 19.09, Python3 is now supported (in addition to Python2).
The TestcaseLet’s say I have a stable verification environment and I want to make it more efficient. For example: I want to check whether I can make the tests shorter while hardly harming the coverage. I am not sure exactly how to attack this task, so a good place to start is to visually analyze the behavior of the coverage on some typical test I chose. The first thing we need to do is to extract the coverage information of the interesting entities. This can be done using the old Coverage API.
Coverage APICoverage API is a simple interface to extract coverage information at a certain point. It is implemented through a predefined struct type named user_cover_struct. To use it, you need to do the following:
The code example below demonstrates these three steps. We chose to extend the end_group() method and we keep the group grade in some local variable. Note that we divide it by 100,000,000 to get a number between 0 to 1 since the grade in this API is an integer from 0 to 100,000,000.
Pass the Data to a Python ModuleAfter we have extracted the group grade, we need to pass the grade along with the cycle and the coverage group name (assuming there are a few) to a Python module. We will take a look at the Python module itself later. For now, we will first take a look at how to pass the information from the e code to Python. Note that in addition to passing the grade at certain points (addVal method), we need an initialization method (init_plot) with the number of cycles, so that the X axis can be drawn at the beginning, and end_plot() to mark interesting points on the plot at the end. But to begin with, let’s have empty methods on the Python side and make sure we can just call them from the e code.
# plot_i.pydef init_plot(numCycles): print (numCycles)def addVal(groupName,cycle,grade): print (groupName,cycle,grade)def end_plot(): print ("end_plot")
And add the calls from e code:
Before running this, note that you need to ensure that Specman finds the Python include and lib directories, and Python finds our Python module. To do this, you need to define a few environment variables: SPECMAN_PYTHON_INCLUDE_DIR, SPECMAN_PYTHON_LIB_DIR, and PYTHONPATH.
The Python Module to Draw the PlotAfter we extracted the coverage information and ensured that we can pass it to a Python module, we need to display this data in the Python module. There are many code examples out there for drawing a graph with Python, especially with matplotlib. You can either accumulate the data and draw a graph at the end of the run or draw a graph interactively during the run itself- which is very useful especially for long runs.
Below is a code that draws the coverage grade of multiple groups interactively during the run and at the end of the run it prints circles around the maximum point and adds some text to it. I am new to Python so there might be better or simpler ways to do so, but it does the work. The cool thing is that there are so many examples to rely on that you can produce this kind of code very fast.
In the example we used, we had two interesting entities: packet and state_machine, thus we had two equivalent coverage groups. When running our example connecting to the Python module, we get the following graph which is displayed interactively during the run.
When analyzing this specific example, we can see two things. First, packet gets to a high coverage quite fast and significant part of the run does not contribute to its coverage. On the other hand, something interesting happens relating to state_machine around cycle 700 which suddenly boosts its coverage. The next step would be to try to dump graphic information relating to other entities and see if something noticeable happens around cycle 700.
To run a complete example, you can download the files from: https://github.com/okirsh/Specman-Python/
Do you feel like analyzing the coverage behavior in your environment? We will be happy to hear about your outcomes and other usages of the Python interface.
Orit KirshenbergSpecman team