Is it Necessary to Improve the Quality of Consumer Electronics?

Fellow blogger Joe Hupcey passed along a link covering the recent launch of the BlackBerry Storm. It seems to follow a familiar trend in consumer electronics: great idea, nice product, but the software seems slow and buggy. People send me these articles as if I'm supposed to do something about the situation. The article mentions a specific corner case where the music volume increases when a call comes in, and how Verizon released a firmware upgrade within the first couple of weeks of device availability. One user talks about the sluggish performance and how performance improved when he "updated the operating system". Maybe the operating system engineers talked to the Software Acceleration division of Green Hills Software.

For me articles like this create many questions and don't provide many answers. Gathering the opinions of users is fine to guide consumers about what to buy, but what I'm really interested in are the causes. It would be interesting to look into the development cycle of the Storm and find out how things are done.

Is the schedule too short? It seems providing patches only days after release means the engineers knew about the issues before the release, but somebody decided to just go ahead and start production anyway and then update the software after customers started using the device.

What kind of verification techniques are used? The corner case with the music volume and the call coming in would seem to be something that an engineer would identify on a test plan and identify to verify. This is the kind of planning we encourage with Enterprise Manager and a test that could be generated with ISX. Even if the verification engineer forgot about it during planning, the situation may have been created using a constrained random approach. Maybe there is no way to create this situation except on the real hardware and most software development is done on a model of the hardware. Behavioral models are usually pretty good at modeling all parts of a device like the Storm except the actual phone call.

Are the troubles primarily integration issues? Many times different groups provide software and when the entire stack is assembled there are corner cases and performance issues that pop up. Are there any tools to analyze performance or any defined metrics that document performance goals?

Is it a case of too much work and not enough people? Many projects just don't have enough manpower. The resource allocation provides just the minimum to write the code and not enough bandwidth to do all the debugging and performance analysis and assumes this work can continue after the first release is done.

As a provider of verification tools, it's interesting to try to find out what other types of tests should have been run to better understand performance and expose bugs well ahead of product release. On the other hand, the solution may be much easier than trying to figure out a more comprehensive solution for more planning, coverage, and automated testing. Maybe the solution is to just stop inserting bugs in the first place. Embedded software expert Jack Ganssle recently wrote a column about how it is much cheaper stop inserting bugs than it is to remove them. It's a good summary of how to approach quality from the start. Maybe the solution is to write code that is much simpler to understand easier to maintain. Timothy Strapko provided 10 tips for writing more maintainable code. Most engineers don't really like to spend time testing and even worse is the dreaded phrase "Code Review" because it's bad enough to test your own code, but even worse to think about spending time trying to understand somebody else's code for the purpose of giving constructive feedback. Sure, double checking your work (or your neighbors) sounds good, but depending on the development culture it may be just a productivity slow down for something engineers don't get much credit for.

We will probably never know all the stories behind the BlackBerry Storm. Maybe consumers would prefer to have the device as soon as possible and don't mind a rough start as the software patches catch up. Maybe software development will always follow an iterative model where we get something out the door and then continue to iterate on it. For example, the Linux kernel has been iterating and improving since 1991. Cadence products are the same, ISX has been iterating for 2 years now with improvements and new features in each release, and other Cadence products for a much longer time. However, consumer electronics seem a bit different, they have a couple year life span and then are thrown away. Maybe the first version of a new architecture is just a test run for the second version to find all the bugs and find out which features need to be added.

Working in an EDA company and discussing embedded software always seems to end with the summary that chip design has a clear cost of failure and very high cost associated with a respin, but software does not have this high cost of failure since new software can be downloaded to a device at any time. It would be very interesting to hear from leaders at Research in Motion or Verizon about how they approach the launch of a new device and the trade-off of time-to-market vs. quality and damage to the brand. Can the quality of consumer electronics be improved? Probably, but does anybody want to improve the quality? Maybe it's not necessary to improve the quality. Maybe there is a fine line between too buggy and just buggy enough that determines the release criteria and maximizes product revenue. Maybe the Storm launch was executed perfectly according to plan at just the right time to capture the maximum market.

Tips on how to improve embedded software quality are always welcome, but insight into trade-off of quality and time-to-market for consumer electronics would be especially welcome.