Posts Tagged ‘programming’

Verification vs. Validation

Tuesday, December 15th, 2009 Michael Barr

The FDA 510(k) guidelines for medical device software leave something to be desired in the poor differentiation of two important and distinct software development practices: verification and validation.  In particular, the FDA often uses the word ‘validation’ to describe both types of activities.  (See, for example, the General Principles of Software Validation; Final Guidance for Industry and FDA Staff.)

Put simply, software validation is a set of activities that together demonstrate that you “made the correct product” (or, as others have put it, “built the right thing”) for the customer’s needs.  Validation tests that the product’s behavior is consistent with the requirements, safe, and efficacious.

By contrast, software verification is a set of activities that together demonstrate that the implementation matches the design.  That is, verification tests that you “made the product correctly” (“built it right”).

In the larger context, verification should come before validation.  It doesn’t make sense to check that the product does what it is supposed to unless you first confirm that it does what you programmed it to.  If it were only the case that the many engineers and organizations that talk about software verification and validation (a.k.a., V&V) could get this simple concept.  It wouldn’t hurt, of course, if the FDA rewrote the above document.

Embedded Programmers Worldwide Earn Failing Grades in C and C++

Tuesday, November 24th, 2009 Michael Barr

In industry surveys, over 80% of embedded software developers report using C or C++ as their primary programming language. Yet as a group, these programmers earned a failing grade on a multiple-choice quiz testing firmware-related C programming skills. A scary result, considering that embedded software inside medical devices, industrial controls, anti-lock brakes, and cockpits place human lives at risk every day.

In a February 2008 blog post, I examined the first few hundred results from the “Embedded C Quiz” on the Netrino website. That analysis compared the performance of programmers in the U.S. and India with the rest of the world (the only three data sets large enough for meaningful analysis). I concluded that the average embedded programmers in the U.S. and India don’t know C very well, but do know it better than programmers in the rest of the world.

Two years now since launching the quiz, we have collected thousands of data points, so it’s time for an update on programmer performance. In total, 3,870 programmers have taken the short 10-question multiple-choice C skills test. A few (a bit less than 3%) didn’t answer all of the questions; the analysis below is based on just the 3,755 completed quizzes. (Note that each website user can only take the quiz once.)

Across all countries, the mean result was 60.8%–a grade of ‘D-‘ at best. That is to say that the average embedded programmer answered just 6 out of 10 multiple-choice questions correctly. A rather scary fact, given that C is the language of choice for most embedded projects and that C++ is even harder to master.

Programmers in the United States scored slightly above average. But they still earned a failing grade of 61.8%. Programmers in India scored slightly below the worldwide average, at 58.9%. Together, programmers from these two large English-speaking countries accounted for the majority of all quiz takers.

The number of completed quizzes, mean scores, and standard deviations for all countries with more than 20 completed quizzes are shown in the table below, sorted by average score. In general, programmers from European countries scored best.

Country Completed Mean Std Dev
Poland 23 68.7 19.2
Sweden 26 67.7 15.8
Australia 45 67.3 22.3
Germany 57 67.2 17.2
France 35 66.9 24.0
United Kingdom 109 66.1 22.8
Spain 24 65.0 18.3
Canada 114 64.5 19.3
China 51 64.1 23.4
Israel 22 62.3 21.7
United States 1346 61.8 20.4
Egypt 28 59.3 22.8
India 1288 58.9 22.4
Romania 45 58.9 23.0
Singapore 24 58.3 20.1
Italy 44 56.4 20.8
Turkey 57 55.6 23.3
Brazil 47 55.1 24.1
Pakistan 25 44.0 21.7

How are your embedded C programming skills? Test them by taking the Embedded C Quiz yourself now at http://www.netrino.com/Embedded-Systems/Embedded-C-Quiz?

P.S.  We recently launched an Embedded C++ Quiz and the results so far look downright abysmal.  I’ll write something about that in a future post.  Do you have a few minutes to take that one too?

Help Bring the Embedded Software Boot Camp to Your City

Friday, October 30th, 2009 Michael Barr

When Netrino announced the first public offering of the Embedded Software Boot Camp a year and a half ago, I had no idea how popular it would be. Or just how much I could love teaching the intensive hands-on week-long version of the training we had developed over many years.

At this point, we have educated hundreds of engineers about embedded software architecture and related best practices through the topics of Hardware Interfacing in C, Multithreaded RTOS Programming, and RTOS Alternatives and ARM-based programming exercises.

Here’s just a small sampling of feedback from recent attendees:

“I would like to thank you again for the Embedded Software Boot Camp. I brought all the books back to the company and showed my boss the slides and all the handouts and all that good stuff and he was very impressed. Needless to say he was happy with the investment he made in Netrino.” — Garrett

“A better use of time and money than the Wind River VxWorks training course I took last month!” — David, IBM

“Hands on exercises are well thought out.” — Mahesh

“This is one of the best trainings I have ever attended.” — H., Hughes Network Systems

“Fabulous, pertinent, comprehensive and articulate collection of the most important things needed practically. Awesome!” — Sourabh

“Complete and correct embedded software training.” — P. Sipika

For 2010, we are planning a multi-city worldwide road-show for this popular event. I plan to teach as many of them personally as I can. I’d love to have you join us, but we first need your input to select the best cities and dates. If you’ve got a minute, please take our quick 5-question online survey at:

http://survey.constantcontact.com/survey/a07e2m7qx6ig1dxfq9v/start

No personally identifying information is gathered in the survey. Thus if you want to be the first to know what cities and dates we choose, be sure to sign up for our mailing list or bookmark our public training calendar.

Logs and the Logging Loggers who Log Them

Wednesday, October 7th, 2009 Michael Barr

There is some excellent concise advice about creating a logging framework for embedded software in the recent MD&DI article “Software Design for Test“.

Portions of the advice are so good, they bear restatement in the context of our ongoing discussion about reliable firmware architecture:

1. Verbosity The API for your logger module should allow for the selection of a verbosity level, within a range from debug-only to critical. That way, application code that desires to have log something need not be changed even as you shift to a production release environment in which you wish to minimize log memory size.

2. Timestamps All logged events should be timestamped. Furthermore, these timestamps should be added as close as possible to the time of the actual event’s occurrence. Note that timer ticks since reset may be an acceptable form of a timestamp in systems not otherwise requiring date/time knowledge.

3. Readability The underlying log format should favor readability by humans. Event logs will ideally be parse-able by automated testing tools as well as by humans who must occasionally develop new automated tests. If the underlying log format cannot itself be human readable, then an offline tool should be provided to translate the raw log data to a human readable format.

If your logger module does only these three things, it will provide a powerful foundation on which much additional functionality can be built.

Binary Literals in C

Wednesday, September 30th, 2009 Michael Barr

A couple of years ago, Netrino engineer Dan Smith was writing stepper motor control firmware that interfaced to lots of registers with binary fields and sub-fields. After struggling a bit with the usual error-prone “off by 1 bit shift” masking and conversion from binary to hexadecimal literals in C, he happened across a useful post on a forum.

In a nutshell, the “binary literal” technique involves the following set of C preprocessor macros:

// Internal Macros
#define HEX__(n) 0x##n##LU
#define B8__(x) ((x&0x0000000FLU)?1:0) \
+((x&0x000000F0LU)?2:0) \
+((x&0x00000F00LU)?4:0) \
+((x&0x0000F000LU)?8:0) \
+((x&0x000F0000LU)?16:0) \
+((x&0x00F00000LU)?32:0) \
+((x&0x0F000000LU)?64:0) \
+((x&0xF0000000LU)?128:0)

// User-visible Macros
#define B8(d) ((unsigned char)B8__(HEX__(d)))
#define B16(dmsb,dlsb) (((unsigned short)B8(dmsb)<<8) + B8(dlsb))
#define B32(dmsb,db2,db3,dlsb) \
(((unsigned long)B8(dmsb)<<24) \
+ ((unsigned long)B8(db2)<<16) \
+ ((unsigned long)B8(db3)<<8) \
+ B8(dlsb))

Here are some examples of the usage of these macros:
B8(01010101) // 85
B16(10101010,01010101) // 43,605
B32(10000000,11111111,10101010,01010101) // 2,164,238,933

So if you had a memory-mapped 8-bit control register of the format XXXYYZZZ (where XXX, YY, and ZZZ are subfields), you could initialize it like so:

*p_reg = ( (B8(010) << 5) | (B8(11) << 3) | (B8(101) << 0) )

which sets the XXX bits to 010, YY to 11, and ZZZ to 101. If I ever needed to change XXX to 011, just change a single 0 to a 1 in the source code, and everything magically changes. Best of all, it’s all done at compile-time. No error-prone conversion to hexadecimal necessary, no figuring out which bits belong to which nibbles, etc.

What is that old saying? — “good programmers write good code; great programmers steal great code