The value of chaos (really!)

12 Jul 2008 • 2 minute read

Ordinarily chaos is bad thing. Yet like it or not, the world your SoC lives in is in complete chaos -- your chip will see all sorts of unexpected stimuli and unimagined use cases over time, which can lead to end users finding the bugs you missed.

However, in this day and age it amazes me how many customers I see who are still only writing directed tests to stimulate their DUT, without using any randomization at all. Yup, I still see places where the testbench does not randomize even a single variable, signal, or vector (just for loop sweeps and such). Why is this bad? IMHO, there are 2 big reasons:

1) Creating stimulus in this way just doesn't scale. No engineer, heck, no army of engineers, can write enough tests to come even close to hitting even the main state spaces of the chips I'm seeing out there today (some with roadmaps to 1 billion *logic* gates in just a few years)

2) You will only find the bugs you were looking for. This point is sort of like the old motorcycle riders' saying, "one of your biggest threats is the car you don't see". (and as any biker knows, there are plenty of ways to be negatively surprised no matter how attentive you are.)

However, rather than fight this chaos, you can actually use it to your benefit. That's right -- chaos is good when you channel it in the right way, and and bring in the chaos from the outside world into your verification in a controlled way with constrained random generation.

Make sense?

Is there any such thing as a "good" directed test?

When (in the stimulus creation context) is chaos and randomness ever a bad thing?

Full disclosure:
Before going to "the dark side" (marketing), in a past life I worked in ASIC verification. I was getting paid good $$$ to write directed tests and compare the output to the golden waveforms, and as the tests and verification environment matured, automate the checking with Perl wrapped around C wrapped around the HDL of choice. In all of my projects, the excitement of the initial ramp-up was soon followed by the drudgery of dreaming up new ways to automate & combine stimulus. Eventually it drove me batty enough to drop out of engineering and go to B-school [Cue the chorus: "Nooooooo!"] Suffice to say if today's languages, tools, and methodologies existed back then I might still be in engineering. [Cue the violins]