In Part 1 of this series on Android System Verification I provided the basics about how to run the Android emulator. When I initially looked at the emulator I was looking for information about available verification techniques, primarily at the system level. I surmised that many Android projects aggregate the available software, add some new custom software to support a specific hardware platform, enable the wide range of available applications, and put all of this together to create a product. Android provides a very high level of design reuse, which is great, but reuse can also be error prone. In fact, hardware verification engineers commonly tell stories about how they pay very close attention to verification on a new design and take nothing for granted. Then, on "derivative" designs less attention is paid to verification (since not much is changing is should take less time) and sometimes issues escape the verification process. Interestingly, the escapes usually have to do with design blocks that were throughly verified and used on previous projects, but when placed into new circumstances had a problem. ISX users also describe how software changes can cause hardware issues to come out in hardware blocks that were fine on the previous project. My hypothesis is that embedded software is the same. Typically it runs very well in some set of circumstances, but when things change around it may not behave as well.
Here are a few testing techniques available in Android:
I'm going to skip over Java unit testing, since my focus is on system verification, not individual blocks of Java code.
Monkey provides a way to generate pseudo-random streams of user events. It is meant to stress test applications, just as if a monkey was pounding on the keyboard or screen (and hopefully not breaking it). Monkey takes a seed as input to a random generator so that the tests are repeatable. It can be applied to either all of the Android software applications or can be constrained to a list of packages or a category of packages. Monkey is actually a program running inside the device (the trade off between generating stimulus inside a system vs. from the outside is a topic for another time). Because it is run on the target system, it is started using the Android Debug Bridge (adb), a tool providing many useful services for interacting with a target device from the host machine.
Monkey is started using adb shell, for example:
% adb shell monkey –v 500 –s 2
will run monkey on all of the software with verbose mode on, random seed 2, and 500 events generated. Monkey appears to be useful for testing application and cheaper than hiring an actual monkey to do the testing.
The emulator console turned out to be more interesting for me because it allows hardware commands to be sent to the system. As I mentioned in Part 1 I feel like big improvements are possible in the hardware stimulus generation and checking that is currently used in Virtual Platforms like the Android emulator. The emulator console can dynamically query and control the simulated device over a telnet connection.
To connect to the console use:
% telnet localhost 5554
Here is the output of the help command, you can read about all the commands in the documentation.
Using the emulator console you can issue a variety of hardware commands, here are the main categories of commands:
Use the documentation details to issue these commands from a telnet session and see things happen on the phone, it's pretty cool. My kids were convinced it was a real phone and somebody was trying to call me. The emulator actually rings when a call comes in.
Here are some screen shots I took of the emulator running.
Of course, the GPS shows I'm located in Minnesota.
Give it a try, and next time we will get into how to use this telnet connection for automated verification.