Way Worse Than The Real Thing

This week Cadence and Virtutech announced a collaborative effort to bring together the Virtutech Simics virtual platform with the Cadence ISX software testing system. This is a very interesting combination of technologies, clearly demonstrating how virtual platforms make it possible to test software in ways that are plain impossible on physical hardware. With a virtual platform, it is possible to systematically subject a system software stack to extreme (and normal) working conditions, trying to provoke errors and find conditions where the software breaks.

We are not just doing regular automated software testing as you would do any host, but also changing parameters for the hardware on which the software runs. In this particular example we are showing at CDNLive! EMEA, we have a time-to-result latency parameter that determines the time it takes for the hardware to issue an interrupt to the software. The software behaviors demonstrated with latencies at 1 ns and 1 s are quite different... in the fast case, the driver software needs to ensure it can handle an interrupt immediately after starting an operation. In the slow case, the driver has to put the calling process to sleep and wake it up later so that the rest of the system can go on working while we wait for the hardware unit to complete. Try doing that on hardware...

Some other obvious examples of how hardware can be controlled to stress software is to vary the number of processors, the speed of the processors, or the size of memory in a system. Isn't it interesting to find out what REALLY happens if you have 2^64 bytes of physical RAM? Will the OS suffer an integer overflow and report -2^63 bytes? Or if all processors run at different clock frequencies? It tends to reveal all kinds of bad assumptions in code. If you add in the ability to inject known bad results from hardware, or turning off units to simulate local hardware failures, you have a very powerful tool to exercise software correctness and robustness. Putting a space-bound computer board in a radiation chamber to check robustness for transient errors is not quite as easy as injecting a sprinkle of memory errors in a simulation.

Thinking about it... this is how hardware testing has always been done: create controlled experiments, often looking for corner cases. For software, that has not really been possible since the execution hardware was always fundamentally uncontrollable. But with a virtual platform, suddently the hardware becomes controllable. And the possibilities become boundless. The Cadence ISX integration with Virtutech Simics is just a first step, and we have only scratched the surface of a very exiting topic. The software that was used to have a nice and cushy environment to run needs to wake up and realize that it will be subject to something worse, way worse, than the real thing.

This Team ESL posting is provided by Dr. Jakob Engblom, Technical Marketing Manager for Virtutech. He holds an MSc in Computer Science and a PhD in Computer Systems from Uppsala University. He has been with Virtutech since 2002, and has worked with embedded software, real-time programming, and computer systems simulation for the past decade.