Archive for the ‘General C issues’ Category

Keeping your EEPROM data valid through firmware updates

Thursday, November 26th, 2009 Nigel Jones

Back when embedded systems used EPROM (no that is not a typo for my younger readers) rather than Flash, the likelihood of the code being updated in the field was close to nil. Today however, it is common for embedded systems to contain mechanisms to allow the code to be updated easily. Like most people, I embraced this feature enthusiastically. However, after I’d implemented a few systems that were field upgradable, I discovered that the ability to update in the field had an unexpected impact on my EEPROM data. To see what I mean, read on…

Most of the embedded systems I work on contain EEPROM. One of the prime uses for this EEPROM is for storing configuration / calibration information for the system. As a result, I often store data in EEPROM as a series of data structures at fixed locations, with gaps in between them. Thus, my EEPROM map might look something like this:

#define CAL_DATA_LOCATION     0x0010
#define CONFIG_DATA_LOCATION  0x0200
...
#define SYSTEM_PARAMS_LOCATION  0x1000
typedef struct
{
 uint32_t param1;
 uint16_t param2;
 ...
 uint8_t  spare[10];
} CALIBRATION_DATA;
__eeprom CALIBRATION_DATA Cal_Data @ CAL_DATA_LOCATION;
__eeprom CONFIGURATION_DATA Config_Data @ CONFIG_DATA_LOCATION;
...
__eeprom SYSTEM_DATA System_Data @ SYSTEM_PARAMS_LOCATION;

As you can see, I was smart enough to allow room for growth within the structure via the spare[] array. (I have intentionally omitted support related to corruption detection to avoid complicating the issue at hand). As a result I thought I was all set if at some time a SW update caused me to have to use more parameters in a given EEPROM structure. Well I went along in this blissful state of ignorance for a few years until the real world intruded in a rather ugly way. Here’s what happened. The firmware upgrade didn’t require me to add any new parameters to the EEPROM, per se, but it did require that the data type of some of the parameters be changed. For example, my CALIBRATION_DATA structure example might have to change to this:

typedef struct
{
 float     param1;
 uint16_t  param2;
 ...
 uint8_t   spare[10];
} CALIBRATION_DATA;

Thus param1 has changed from a uint32_t type to a float. Thus when the new code powered up, it had to read param1 as a uint32_t, and then convert it to a float type and write it back to the EEPROM. This clearly was quite straightforward. However, where the problem came was the next time the system powered up. I realized that without some sort of logic in place, I would re-read param1, treat it as a uint32_t (even though it is a float), ‘convert’ it to a float and write it back to EEPROM. Clearly I needed some method of signaling that I had already performed the requisite upgrade. As I pondered this problem, I realized that it was even more complicated. Let us denote the two versions of CALIBRATION_DATA as version 1 and version 2 respectively. Furthermore, let’s assume that in version 3 of the code, param1 gets changed to a double (thus shifting all the other parameters down and consuming some of the spare allocation). I.e. it looks like this:

typedef struct
{
 double     param1;
 uint16_t   param2;
 ...
 uint8_t    spare[6];
} CALIBRATION_DATA;

In this case, we must not only be able to handle the upgrade from version 2 to version 3 – but also directly from version 1 to version 3. (You could of course require that users perform all upgrades in order. While I recognize that sometimes this is unavoidable, I suspect that most times it’s because the developer has backed themselves in to the sort of corner I describe here).

Anyway, with this insight in hand, I realized that I needed a generic system for both tagging an EEPROM structure with the version of software that created it, together with a means of providing arbitrary updates. This is how I do it.

Step 1.
Make the first location of each EEPROM structure a version field. This version field contains the firmware version that created the structure. By making it the first location in the EEPROM data structure, you ensure that you can always read it regardless of what else happens to the structure. Thus my CALIBRATION_DATA structure now looks something like this:

typedef struct
{
 uint16_t version;
 uint32_t param1;
 uint16_t param2;
 ...
 uint8_t  spare[10];
} CALIBRATION_DATA;

Step 2.
Add code to handle the upgrades. This code must be called before any parameters are used from EEPROM. The code looks something like this:

void eeprom_Update(void)
{
if (Cal_Data.version != SW_VERSION)
{
 switch (Cal_Data.version)
 {
  case 0x100:
   /* Do necessary steps to perform upgrade */
  break;
  case 0x200:
   /* Do necessary steps to perform upgrade */
  break;
  default:
  break;
  Cal_Data.version = SW_VERSION;  /* Update the EEPROM version number */
 }
}

Incidentally, I find that is often one of those cases where falling through case statements is really useful. Of course doing this is usually banned and so one ends up with much more clumsy code than would otherwise be required.

An Apology
Regular readers will no doubt have noticed that this is my first post in a while. A deadly combination of vacation and urgent projects with tight deadlines had conspired against me to prevent me blogging at my usual pace.

Home

Eye, Aye I!

Friday, November 6th, 2009 Nigel Jones

Today’s post should probably be called ‘Thoughts on non-descriptive variable names’, but once in a while I have to let my creative side out!

Anyway, the motivation for today’s post, is actually Michael Barr’s latest blog posting concerning analysis of the source code for a breathalyzer. Since I do expert witness work, as well as develop products I was keen to see what the experts in this case had to say. One snippet from the expert for the plaintiffs caught my eye. In appendix B of their report, Draeger made the following statement concerning general code issues:

Non descriptive variable names – i, j, dummy and temp

This touched upon something where I seem to be at odds with the conventional wisdom. I’ll illustrate what I mean. Consider initializing an array to zero (I’ll ignore that we could use a library function for this). I would code it like this:

uint8_t buffer[BUFSIZE];
uint8_t i;
for (i = 0; i < BUFSIZE; i++)
{
 buffer[i] = 0;
}

This code would be rejected by many coding standards (and apparently would offend Draeger), as the loop variable ‘i’ is not descriptive. To be ‘correct’, I should instead code it like this

uint8_t buffer[BUFSIZE];
uint8_t buffer_index;
for (buffer_index = 0; buffer_index <BUFSIZE; buffer_index++)
{
 buffer[buffer_index] = 0;
}

So for me, the question is, does the second approach buy me anything – or indeed cost me anything? Well clearly, this is a matter of opinion. However I’d make the following observations:

  1. I think my code is clear, concise and easily understood by even the most unskilled programmer
  2. Is the variable name ‘buffer_index’ clearer – yes but only to a native English speaker. It’s my experience that there are a lot of non-native English speakers in the industry.
  3. Personally, I find the use of similar words in close proximity (buffer[buffer_index]) to be a bit harder to read, and very easy to mis-read if there are other variables around prefixed with buffer.

I’d also make the observation that many coding standards require variable names to be at least 3 characters long, and as a result I’ve seen code that looks like this:

uint8_t buffer[BUFSIZE];
uint8_t iii;
for (iii = 0; iii < BUFSIZE; iii++)
{
 buffer[iii] = 0;
}

Clearly in this case, the person is addressing the letter of the standard (if you’ll pardon the pun), but not the spirit. Where the standard requires the variable names to be meaningful, I’ve also seen this done:

uint8_t buffer[BUFSIZE];
uint8_t idx;
for (idx = 0; idx < BUFSIZE; idx++)
{
 buffer[idx] = 0;
}

This code meets the letter of the standard, and arguably the spirit. Is it really any more understandable than my original code? I don’t think so – but I’ll be interested to get your comments.

Home

Whither white space?

Tuesday, October 20th, 2009 Nigel Jones

I was looking over some code I wrote a year ago in preparation for making some minor enhancements, when I noticed that in one place I had two blank lines between functions, instead of my coding standard mandated one line. I immediately and instinctively corrected it – as is my norm. However having done so, I paused to consider what I’d just done – and why.

On the one hand, this was a clear violation of my coding standard – and so it must be corrected. However, the violation wasn’t doing any harm, per se, and correcting it came at a cost – namely that someone (probably me) browsing the version control system at a later date will see that the file has been touched – and may choose to investigate what was changed – only to find out that it was a simple white space correction. (I appreciate that version control systems can be set up to ignore white space. I choose to not use that option).

Now I suspect that readers of this blog will be divided. Some will think I was quite right to eliminate the extra line, whereas others are thinking – doesn’t this guy have better things to do in life? Which brings me to my point!

Some people are completely anal retentive when it comes to white space. They are very careful on indentation, alignment of comments, use of blank lines and so on. I fall squarely into this category – as does Jean Labrosse of MicroOS II fame. Others could not care less about white space. They will arbitrarily have 6 blank lines between two functions,and then no lines between the next two functions. Their comments are usually aligned all over the place, and they rarely use space between e.g. the elements of a for loop statement. Finally, there’s the third (and largest group) who fall somewhere in between these two extremes.

Now I look at a lot of code, and  having done so, I think I can make a sweeping generalization, which I’ll call the “Nigel Jones white space principle”. Succinctly put, it states:

White space discipline is highly correlated with coding discipline.

That is, those who are careless about white space are often careless about a lot of other things. The converse seems to apply. As a result, when I look at code, literally the first thing I note is how well disciplined was the author in the use of white space. If the code is cleanly and consistently laid out, then I get a good first impression, and the chances are the code will be first rate.

Now I am unsure which is the cause and which is the effect here. In other words, does white space discipline lead to more disciplined code overall, or is it the other way around? Regardless, if your code looks like a mess, then I’d humbly suggest that you literally clean up your act – your career will thank you!

Home

Is MISRA compliance worthwhile?

Wednesday, October 7th, 2009 Nigel Jones

I had been planning on talking about MISRA C compliance in a month or two from now. However, in the comments section of my recent post about bitfields, Anand posed the following question:

However I am writing to ask your opinion about MISRA C compliance. This is the first time a client company has asked us for MISRA C compliance and I am not quite sure where to start. I started reading the guidelines from the web, however soon realised that its an enormous task and I would never get through all of them. So what I would like to know from you is how realistic is it to expect to comply with all the guidelines? Since my compiled code size is less than 2KB so should we even bother with MISRA C compliance?
Your valuable insights would be highly appreciated

I first gave my thoughts about MISRA C in an article published in 2002 in Embedded Systems Programming magazine (now Embedded Systems Design). Looking back over the article, I don’t see anything in it that I disagree with now. However, there are certainly some things that I’d add, having attempted to adhere to the MISRA guidelines in several large projects. After I have given my thoughts, I’ll try and address Anand’s questions.

I’ll start by noting that since I wrote the aforementioned article, MISRA released a second edition of their guidelines in 2004. The second edition was a major revision, and attempted to address many of the ambiguities in the 1998 version. As such, if someone is asking for MISRA compliance, they usually mean the 2004 rules; however it would behoove you to check!

Most of the MISRA rules can be checked via static analysis (i.e. by a compiler like tool) and indeed many more compilers now come with a MISRA checking option. Thus for those of you that are using such a compiler, conformance to most of the rules may be checked by simply using the correct compiler switch. For those of you that don’t have such a compiler, a great alternative is my favorite tool – PC-Lint from Gimpel – which brings me nicely to the main point I wish to make. MISRA  attempts to protect you from the darkest, nastiest corners of the C language – which is exactly what PC-Lint attempts to do as well. However, MISRA C attempts to do it by banning various constructs, whereas PC-Lint attempts to detect and inform you when you are using a construct in a potentially dangerous way. To put it another way, MISRA treats you like a child and PC-Lint treats you like an adult. As a result, I’ll take code that is ‘Lint free’ over code that is ‘MISRA compliant’ any day. I’d also add that making code Lint free is often a lot more challenging than making it MISRA compliant.

Now does this mean that I think the MISRA rules are not worthwhile? Absolutely not! Indeed the vast majority of the rules are pretty much good programming practice in a codified form. For example, rule 2.2. states: “Source code shall only use ISO9899:1990 ‘C’ style comments”. Now regardless of whether you agree with this rule or not, it’s my opinion that source code that contains C style comments and C++ style comments reflects a lack of discipline on the author’s behalf. Thus I like this rule – and I adhere to it.

Where I start to run into problems with MISRA are rules such as 20.6 “The macro offsetof in stddef.h shall not be used”. I wrote an article in 2004 for Embedded Systems Programming magazine entitled “Learn a new trick with the offsetof() macro”. The examples I give in the article are elegant and robust solutions to certain common classes of problems in embedded systems. Solving these problems without using the offsetof() macro is hard and /or tedious and / or dangerous. In short the medicine prescribed by MISRA is worse than the supposed disease.

Putting this all together, my feelings on MISRA are as follows.

  1. It’s intentions are excellent – and I wholeheartedly support them.
  2. Most of its rules are really good.
  3. There are times when I just have to say – sorry your attempts to make my code ‘safer’ are actually having the opposite effect and so as an experienced embedded systems engineer I’m choosing to ignore the rule in the interest of making a safer product. Note that I don’t do this on a whim!

Now clearly, when it comes to my third point, I can certainly be accused of hubris. However, at the end of the day my clients typically hire me for my experience / knowledge and not for my ability to follow a rule book.

As a final note, I must say that I think the MISRA committee has overall done a very fine job. Trying to come up with a set of rules for the vastly disparate embedded systems industry (I know they are only really aimed at the car industry) is essentially an impossible task.

So with that all of the above as a preamble, I think I can address Anand’s questions.

Where to Start?
Order a copy of the guidelines from MISRA. You can get an electronic copy for £10 (about $15). If you are using a C compiler that has a MISRA checking switch, then turn it on. If not buy a copy of PC-Lint. If you are not already using PC-Lint then you should be. It will be the best $389 you ever spent.

Then What?
Take a snapshot of your code base in your version control system before starting.
The first time you check your code for compliance, you will undoubtedly get an enormous number of errors. However, I think you will find that half a dozen rules are creating 90% of the problems. Thus you should be able to knock the error count down fairly quickly to something manageable. At that point you will be in to some of the tougher problems. My recommendation is not to blow off the MISRA errors, but rather to understand why MISRA thinks your constructs are unsafe. Only once you are convinced that your particular instance is indeed safe should you choose to ignore the violation.

Is it worth it for 2K of object code?
Yes – and no. With an executable image of 2K, the chances are most of your code is very tightly tied to the hardware. In my experience, the closer you get to the hardware, the harder it is to achieve MISRA compliance, simply because interaction with the hardware typically relies upon extensions to standard C – and these extensions violate MISRA rule 1.1. Thus you have no hope of making your code compliant, literally starting with the first rule. (The MISRA committee aren’t stupid, and so they have a formal method for allowing you to waive certain rules – and this is a clear example of why it’s necessary. However, it’s a tough way to start out). Does this mean that the exercise is pointless though? Probably not, as at the end of the day you’ll probably have cleaner, more portable, more easily maintained code. However, I seriously doubt that it will be compliant.

Finally, I’ll answer a question that you didn’t ask. What should I say to a client that asks for MISRA compliance? Well the first thing to determine is whether compliance is desired or required. If it’s the former, then what they are probably asking for is work that conforms to industry best practices. In which case what I normally do is explain to them that the code I deliver will be Lint free – and that this is a far higher hurdle to cross than MISRA compliance. If however MISRA compliance is a must, then you have no option other than to bite the bullet and get to work. It would probably make sense to retain a consultant for a day or two to help you get up to speed quickly.

Home

Minimizing memory use in embedded systems Tip #3 – Don’t use printf()

Thursday, September 24th, 2009 Nigel Jones

This is the third in a series of tips on minimizing memory consumption in embedded systems.

If you are like me, the first C program you saw was K&R’s famous ‘hello, world’ code, reproduced below:

main()
{
 printf(“hello, world\n”);
}

In my opinion, this program has done incalculable harm to the realm of embedded systems programming! I appreciate that this is a rather extreme statement – but as is usual I have my reasons …

The interesting thing about this code is that it introduces printf() – and as such gives the impression that printf() is an important (and useful) part of the C language. Well I suppose it is / was for those programming computers. However for those programming embedded systems, printf() and its brethren (sprintf, vsprintf, scanf etc) are in general a disaster waiting to happen for the unwary. Here is why:

Code Size

The printf() functions are immensely sophisticated functions, and as such consume an incredible amount of code space. I have clear memories of an early 8051 compiler’s printf() function consuming 8K of code space (and this was at a time when an 8K program was a decent size). Since then, compiler vendors have put a lot of effort into addressing this issue. For example IAR allows you to specify the functionality (and hence size) of printf() as a library option. Notwithstanding this, if your available code space is less than 32K the chances are you really shouldn’t be using printf(). But what if you need some of the features of printf()? Well in that case I recommend you write your own formatting function. For example I often find that I have a small microcontroller project that needs to talk over a serial link using an ASCII protocol. In cases like these, the easy thing to do is to generate the requisite string using a complex format string with sprintf(). However, with a little bit of ingenuity you should be able to create the string using a series of calls to simple formatting routines. I can guarantee that you’ll end up with more compact code.

Stack Size

Barely a day goes by that someone doesn’t end up on this blog because they have a stack overflow caused by printf(), sprintf() or vsprintf(). Why is this? Well if you are ever feeling bored one day, try and write the printf() function. If you do, you’ll soon find that it is not only difficult, but also that it requires a large amount of space for the function arguments, a lot of temporary buffer space for doing the formatting as well as a large number of intermediate variables. In short, it needs a tremendous amount of stack space. Indeed I have had embedded systems that need a mere 32 bytes of stack space prior to using printf() – and 200+ bytes after I’ve added in printf(). The bottom line is that for small embedded systems, formatted output needs a ridiculous amount of stack space – and that as a result stack overflow is a real possibility.

Variable length arguments

I’m sure most people use sprintf() etc without fully appreciating that these functions use a variable length argument list. I’ll leave for another day the full implications of this. However for now you should just consider that MISRA bans the use of variable length arguments – and that you should take this as a strong hint to avoid these functions in embedded systems.

Execution time

The execution time of printf() can be spectacularly long. For example the ‘hello world’ program given in the introduction requires 1000 cycles on an AVR CPU. Changing it to the almost as trivial function shown below increases the execution to 6371 cycles:

int main( void )
{
 int i = 89;

 printf("hello, world %d\n", i);
}

Lest you think this is an indictment of the AVR processor, the same code for a generic ARM processor still takes a whopping 1738 cycles. In short, printf() and its brethren can take a really long time to execute.

Now do the above mean you should always eschew formatted output functions? No! Indeed I recommend the use of vsprintf() here for certain classes of problem. What I do recommend is that you think long and hard before using these functions to ensure that you really understand what you are doing (and getting) when you use them.

Previous Tip
Home