embedded software boot camp

Breathalyzer Source Code Analysis

November 5th, 2009 by Michael Barr

Firmware bugs seem to be everywhere these days. So much so that firmware source code analysis is even entering the courtroom in criminal cases involving data collection devices with software inside. Consider the precedent-setting case of the Alcotest 7110. After a two-year legal fight, seven defendants in New Jersey DUI cases successfully won the right to have their experts review the source code for the Alcotest firmware.

The state and the defendants both ultimately produced expert reports evaluating the quality of the firmware source code. Though each side’s experts reached divergent opinions as to the overall code quality, several facts seem to have emerged as a result of the analysis:

– Of the available 12-bits of A/D precision, just 4-bits (most-significant) are used in the actual calculation. This sorts each raw blood-alcohol reading into one of 16 buckets. (I wonder how they biased the rounding on that.)
– Out of range A/D readings are forced to the high or low limit. This must happen with at least 32 consecutive readings before any flags are raised.
– There is no feedback mechanism for the software to ensure that actuated devices, such as an air pump and infrared sensor, are actually on or off when they are supposed to be.
– The software first averages the initial two readings. Then it averages the third reading with that average. Then the fourth reading is averaged in, etc. No comments or documentation explains the use of this formula, which causes the final reading to have a weight of 0.5 in the final value and the one before that to have a weight of 0.25, etc.
– Out of range averages are forced to the high or low limit too.
– Static analysis with lint produced over 19,000 warnings about the code (that’s about three errors for ever five lines of source code).

What would you infer about the reliability of a defendant’s blood-alcohol level if you were on that jury? If you’re so inclined you can read the full expert reports for yourself: at defendants and state.

Tags: , , ,
Posted in Uncategorized | 6 Comments »

This Code Stinks! The Worst Embedded Code Ever

November 5th, 2009 by Michael Barr

At the Embedded Systems Conference Boston in September, I gave a popular ESC Theater talk titled “This Code Stinks! The Worst Embedded Code Ever” that used lousy code from real products as a teaching tool. The example code was gathered by a number of engineers from a broad swath of companies over several years. (Minor details, including variable names and function names, were changed as needed to hide the specifics of applications, companies, or programmers.)

Here’s just one example of the bad code in that presentation:

y = (x + 305) / 146097 * 400 + (x + 305) % 146097 / 36524 * 100 + (x + 305) % 146097 % 36524 / 1461 * 4 + (x + 305) % 146097 % 36524 % 1461 / 365;

I don’t know if the above snippet contains any bugs, as most of the other examples were found to. And that’s a problem. Where are we supposed to begin an analysis of the above? What is this code supposed to do when it works? What range of input values is appropriate to test? What are the correct output values for a given input? Is this code responsible for handling out of range inputs gracefully? In the original listing, there were no comments on or around this line to help.

I eventually learned that this code computes the year, with accounting for extra days in leap years, given the number of days since a known reference date (e.g., January 1, 1970). But I note that we still don’t know if it works in all cases; despite it being present in an FDA-regulated medical device. I note too that the Microsoft Zune Bug was buried in a much better formatted snippet of code that performed a very similar calculation.

Here’s another example, this time in C++, with the bug finding left as an exercise for the reader:

bool Probe::getParam(uint32_t param_id, int32_t idx)
{
int32_t val = 0;
int32_t ret = 0;

ret = m_pParam->readParam(param_id, idx, &val);

if (!ret)
{
logMsg(“attempt to read parameter failed\n”);
exit(1);
}
else …

Hint: This code was embedded in a piece of factory automation equipment.

I’ve placed the full set of slides online at http://bit.ly/badcode.

Tags: , , , , ,
Posted in Uncategorized | 3 Comments »

TechBites – A Collaborative Community for Engineers

November 3rd, 2009 by Michael Barr

Since at least as far back as January 1999, when I wrote a white paper describing the idea to some colleagues, I’ve wanted to participate in a collaborative community for embedded systems design engineers. I felt then, as I do now, that the collective wisdom spread across the community is largely untapped. This feeling was confirmed when I served as editor-in-chief of Embedded Systems Programming magazine, where I tried to turn bits and pieces of your knowledge into articles.

But what if there were a way to reap the benefits of both any willing contributor and an editor able to clean the conversation up, keep things organized, and move threads along? Finally, there is.

A new site launched today called TechBites (http://www.techbites.com) aims to provide a Web 2.0 collaborative community for engineers, where every site member can contribute to the conversation. Yet the site has an impressive list of industry experts already signed up as editors for specific topic-focused communities (e.g., I’ll be helping out with the RTOS and Embedded Medical communities).

At TechBites.com you can:

* Read content including Blogs, Articles, Reviews, and Tech-Tips.
* Create your own content and participate in group discussions.
* Connect with your peers and industry experts.

Join now (it’s free) to take advantage of these opportunities and connect with your friends!

Participate in one or more of the managed communities or start your own user community around a product or technology you know well.

Tags: ,
Posted in Uncategorized | No Comments »

Help Bring the Embedded Software Boot Camp to Your City

October 30th, 2009 by Michael Barr

When Netrino announced the first public offering of the Embedded Software Boot Camp a year and a half ago, I had no idea how popular it would be. Or just how much I could love teaching the intensive hands-on week-long version of the training we had developed over many years.

At this point, we have educated hundreds of engineers about embedded software architecture and related best practices through the topics of Hardware Interfacing in C, Multithreaded RTOS Programming, and RTOS Alternatives and ARM-based programming exercises.

Here’s just a small sampling of feedback from recent attendees:

“I would like to thank you again for the Embedded Software Boot Camp. I brought all the books back to the company and showed my boss the slides and all the handouts and all that good stuff and he was very impressed. Needless to say he was happy with the investment he made in Netrino.” — Garrett

“A better use of time and money than the Wind River VxWorks training course I took last month!” — David, IBM

“Hands on exercises are well thought out.” — Mahesh

“This is one of the best trainings I have ever attended.” — H., Hughes Network Systems

“Fabulous, pertinent, comprehensive and articulate collection of the most important things needed practically. Awesome!” — Sourabh

“Complete and correct embedded software training.” — P. Sipika

For 2010, we are planning a multi-city worldwide road-show for this popular event. I plan to teach as many of them personally as I can. I’d love to have you join us, but we first need your input to select the best cities and dates. If you’ve got a minute, please take our quick 5-question online survey at:

http://survey.constantcontact.com/survey/a07e2m7qx6ig1dxfq9v/start

No personally identifying information is gathered in the survey. Thus if you want to be the first to know what cities and dates we choose, be sure to sign up for our mailing list or bookmark our public training calendar.

Tags: , , , ,
Posted in Uncategorized | No Comments »

Logs and the Logging Loggers who Log Them

October 7th, 2009 by Michael Barr

There is some excellent concise advice about creating a logging framework for embedded software in the recent MD&DI article “Software Design for Test“.

Portions of the advice are so good, they bear restatement in the context of our ongoing discussion about reliable firmware architecture:

1. Verbosity The API for your logger module should allow for the selection of a verbosity level, within a range from debug-only to critical. That way, application code that desires to have log something need not be changed even as you shift to a production release environment in which you wish to minimize log memory size.

2. Timestamps All logged events should be timestamped. Furthermore, these timestamps should be added as close as possible to the time of the actual event’s occurrence. Note that timer ticks since reset may be an acceptable form of a timestamp in systems not otherwise requiring date/time knowledge.

3. Readability The underlying log format should favor readability by humans. Event logs will ideally be parse-able by automated testing tools as well as by humans who must occasionally develop new automated tests. If the underlying log format cannot itself be human readable, then an offline tool should be provided to translate the raw log data to a human readable format.

If your logger module does only these three things, it will provide a powerful foundation on which much additional functionality can be built.

Tags: ,
Posted in Uncategorized | 2 Comments »

« Older Entries
Newer Entries »