Android System Verification Part 4

Welcome to Part 4 of Android System Verification. If you are just joining make sure to read the previous three parts to get up to speed.

In Part 3 I demonstrated how to use a C program to connect to the Android emulator's telnet server and how to write a simple main() function using the C API to send commands to the Android emulator to perform various hardware operations. This was much improved compared to typing commands into a telnet client.

One option for verification is to make the main into a C function containing a test and then create a whole bunch of similar C functions with different types of calls, different parameters, and different order. Given enough time, many C tests could be developed, compiled, and run. You might guess this is not the approach I will describe, not only because the series would end abruptly, but because this would be a very tedious task without knowing if I could think of enough possible tests. Instead, I will show how to achieve better results in less time. There are multiple ways to achieve this, but I'm doing it using Specman and an e Verification Component (eVC).

I face a challenge because some readers are probably expert verification engineers using e and Specman for hardware verification. If you fall into this camp hopefully you will get a glimpse into how system verification might change in the future as abstraction increases. Some readers may be here because they are interested in Android and have no idea what the e language is. I apologize in advance that I cannot provide a proper introduction to verification with e, but a few highlights will have to suffice for now.

The e language is a domain specific language designed for verification, and it has been standardized as a verification language by the IEEE as IEEE 1647. The language uses Aspect Oriented Programming (AOP), a very cool concept to most computer scientists. It's lego-like “extend” increases reuse and results in less code, as well as less editing of existing code when adding new tests. One of the basic principles of e is that everything is is random unless otherwise specified by a constraint. At first, this is tricky to grasp for C programmers because leaving C variables uninitialized is usually not a good idea. The general concept is to use constrained random stimulus to generate tests beyond what even the test writer could think about or list in a test plan. The stimulus is combined with a coverage model and checking to measure what the stimulus achieved and check for any errors along the way. That will have to do for now, but there are many additional resources for those more interested in e. I always recommend the interesting papers and talks by Specman developer Matan Vax, you can access his talk from CDNLive! 2009 Silicon Valley (requires a login on cadence.com).

To get started I will turn the C program into a shared library and show how to automate the stimulus generation using sequences. This will accomplish the first verification goal, automated stimulus generation on the external interfaces of the simulated Android device. The overall concept is shown in the picture below.

To turn the C program into a shared library we can just remove the main and change the compile flags on the same C source file presented last time.

% gcc -o android_console-so -m32 -Wall -shared -g android_console.c -lpthread

Now we have a .so file can that be used by Specman.

The primary task is to create an eVC that will provide automated stimulus generation to the Android emulator. There are lots of ways to create the eVC. One way is to start from scratch and use an editor to code it. Another is to start from an example provided in a Cadence software release or training material. Another way is an eVC generator such as the one provided by the CFS-vision project and make the remaining edits. I used the Incisive Verification Builder (ivb) to create the eVC. Some of you probably know that ivb is a tool to create eVCs and do other tasks like configuring eVCs, but our application here is somewhat unconventional because our eVC will not connect to an HDL simulation with busses and signals, it will connect to the Android emulator. Because of this I used the Incisive Software Extensions (ISX) ivb wizard to create the eVC and then adapted the output of ivb to come up with the needed eVC. The ISX wizard actually creates eVCs that connect to embedded software. This is not embedded software, but our C code to drive the hardware is software so it's not too far off. There are lots of ways to create an eVC. The means is not important and you may even have your own tools and scripts to do it.

In eRM or OVM training we are taught that a Verification Component should have a bus functional model (BFM) that takes care of the signal level handshaking with the Design Under Verification (DUV) and that the BFM should be separated from the stimulus generation. For the application of Android System Verification, the BFM will consist of the C calls that will talk to the Android emulator and we already have this done.

Before we get into the details of the code, some background on sequences is useful. Sequences provide a standardized mechanism to create, control, and drive input streams into a design (in this case the hardware interfaces of the Android device). It will take some initial work to create the environment, but once it's created, test cases can be created that are selections of parameterized sequences that will be sent to the Android emulator. Most important of all, is that unlike the directed C program we used last time, the sequence data items and order of actions are easily randomized within constraints so we can create lots of interesting stimulus very easily.

The ivb generated eVC creates a sequence item called sw_item and a sequence called sw_sequence as shown here:

<'
package aconsole_evc;

struct sw_item like any_sequence_item {
};

sequence sw_sequence using
  item=sw_item,
  created_kind=sw_sequence_kind_t,
  created_driver=sw_sequence_driver_u;

extend sw_item {
  kind : sw_sequence_kind_t;
  activate()@driver.clock is undefined;
};

... more code omitted ...

Next, a sequence library is created by adding a sequence type for each of the hardware operations we looked at last time in the C program:

extend sw_sequence_kind_t: [INIT, CONNECT, GEO, POWER, NETWORK, GSM, SMS];

Each sequence type is a short piece of e code that calls the C function. The code uses the "foreign dynamic C routine" syntax to reference a C function name in a .so file. Here is the sequence to call the C function ac_init() to get the emulator connection established:

extend INIT sw_sequence {
  
   do_init_C() is foreign dynamic C routine android_console:ac_init;
  
   do_init() is {
      do_init_C();
      messagef(HIGH, "called ac_init C function");
   };
  
   body() @driver.clock is {
      do_init();
   };
};

The sequence to set the network speed and delay is:

extend NETWORK sw_sequence {
   net_cmd: network_cmd_t;
   speed: network_speed_t;
   net_del: network_delay_t;

   do_network_C(cmd: network_cmd_t, ops: string) is foreign dynamic C routine android_console:do_network;
  
   do_network(cmd: network_cmd_t, ops: string) is {
      do_network_C(cmd, ops);
      messagef(HIGH, "called do_network C function");
   };

   set_speed(speed: network_speed_t): string is {
      case  speed {
          NET_SPEED_EDGE:  { result = "edge";};
          NET_SPEED_FULL:  { result = "full";};
          NET_SPEED_GPRS:  { result = "gprs";};
          NET_SPEED_GSM:   { result = "gsm";};
          NET_SPEED_HSCSD: { result = "hscsd";};
          NET_SPEED_HSDPA: { result = "hsdpa";};
          NET_SPEED_UMTS:  { result = "umts";};
      };
   };

   set_delay(del: network_delay_t): string is {
      case  del {
          NET_DELAY_EDGE: { result = "edge"; };
          NET_DELAY_GPRS: { result = "gprs"; };
          NET_DELAY_NONE: { result = "none"; };
          NET_DELAY_UMTS: { result = "umts"; };
      };
   };

   body() @driver.clock is {
      var ops: string;

      case net_cmd {
         NET_DELAY: {
            ops = set_delay(net_del);
         };
         NET_SPEED: {
            ops = set_speed(speed);
         };
      };

      do_network(net_cmd, ops);
   };

};

Similar sequences were created for each C function.

Using the sequence library it's easy to write tests ranging from a specific test to a very random test.

Here is a basic test to simply make a call, accept it, and then hangup (3 times):

extend MAIN sw_sequence {
  
   !init_cmd:    INIT        sw_sequence;
   !connect_cmd: CONNECT     sw_sequence;
   !gsm_cmd:     GSM         sw_sequence;

   body() @driver.clock is only {
      var gsm_ops: string;
      var phone_number: string = "4088943000";
     
      // initialize
      do init_cmd;

      // just call, accept, and hangup
      for i from 0 to 2 do {

         outf("calling phone number %s\n",phone_number);

         do gsm_cmd keeping {
            .gsm_cmd in [GSM_CALL];
            .ops == phone_number;
         };

         do gsm_cmd keeping {
            .gsm_cmd in [GSM_ACCEPT];
            .ops == phone_number;
         };

         do gsm_cmd keeping {
            .gsm_cmd in [GSM_CANCEL];
            .ops == phone_number;
         };

      };
   };
  
};

A more random test to pick a sequence with an even distribution from among 5 types and call it using random or default parameter ranges is:

extend MAIN sw_sequence {
  
   !init_cmd:    INIT     sw_sequence;
   !geo_cmd:     GEO      sw_sequence;
   !power_cmd:   POWER    sw_sequence;
   !network_cmd: NETWORK  sw_sequence;
   !gsm_cmd:     GSM      sw_sequence;
   !sms_cmd:     SMS      sw_sequence;
  
   !next_seq : [GEO, POWER, NETWORK, GSM, SMS];
   keep soft next_seq == select {
      20 : GEO;
      20 : POWER;
      20 : NETWORK;
      20 : GSM;
      20 : SMS;
   };
  
   body() @driver.clock is only {
     
      do init_cmd;
     
      for i from 1 to 50 do {
         gen next_seq;
        
         case  next_seq {
            GEO: {
               do geo_cmd keeping {
                  .geo_cmd == GEO_LOCATION;
               };
            };
            POWER: {
               do power_cmd;
            };
            NETWORK: {
               do network_cmd;
            };
            GSM: {
               do gsm_cmd;
            };
            SMS: {
               do sms_cmd;
            };
           
         };
      };
   };
  
};

Now many different tests can be created using the sequence library and a whole bunch of different stimulus can be created very easily. With Specman, the tests can be interpreted (vs. compiled) so it's easy to load a new test and make sure it works as designed. There is random stability using a seed as well as commands to view the generated sequences such as

Specman> trace seq
Specman> show seq

A screen shot of the stripe chart for some generated sequences is below:

 

This gives a pretty good introduction about how to use the C library more effectively by combining it with an eVC for sequence generation.

Next time we will look at how to use coverage to see what was actually done.

Jason Andrews