Archive for the ‘Consulting’ Category

Two out of three ain’t bad

Sunday, March 25th, 2012 Nigel Jones

With all due apologies to Meatloaf for the title, I thought I’d pass along something that I’ve found useful over the years. Being a consultant, I regularly find myself in discussions with clients concerning new product development. Without fail, the following three topics are always high on the agenda:

  1. Features, reliability, bug rates etc. I tend to lump all these into the ‘good’ requirement.
  2. Timescales. When can you start, and more importantly, when can you deliver? I call this the ‘fast’ requirement.
  3. Costs. What will it cost and what can be done to minimize these costs? I call this the ‘cheap’ requirement.

What I tell my clients is this:

You can have it good, you can have it fast and you can have it cheap – but you can’t have all three.

Despite the slightly flippant nature of this meme, I’ve found that clients instinctively recognize the truth in this statement. As a result it forces them to think seriously about what their real priorities are – and to plan accordingly.

Incidentally, you may wonder what combinations end up getting asked for the most. Here’s my qualitative assessment:

Good and Fast

This is the most common. Typically the client that is in a time crunch to deliver something that has already been promised. In situations like this, they have to accept that development is going to be more costly.

Good and Cheap

This is also common. This is the client that is planning ahead. They want a product developed, but know it takes time – and accept this reality.

Fast and Cheap

This comes up when clients are looking for a prototype / proof of concept.

Of course you always get the person that wants all three. I just tell them that two out of three ain’t bad :-).

Why you really shouldn’t steal source code

Saturday, February 11th, 2012 Nigel Jones

As an embedded systems consultant, I spend a substantial part of my work time working on your typical embedded systems projects. However I also spend a significant amount of time working as an expert witness in legal proceedings. While the expert witness work is quite varied, one of the things I have noticed in the last few months is an increase in the number of cases related to source code theft. Typically these cases involve the plaintiff claiming that the defendant has stolen their source code and is using it in a competing product. These claims are often plausible, because as we all know, it’s trivial to walk out of a company with Gigabytes of information in your pocket. Even in companies with strong security measures, it’s normally the case that the engineers are smart enough to work out how to bypass the security systems, so the ‘there’s no way I could have got the code out of there’ defense isn’t usually very plausible.

Thus given how easy it is to steal source code, why shouldn’t you do it? Well let’s start with the obvious – it’s wrong. if you don’t understand this, go and have a chat with your mother – I’m sure she’ll spell it out for you. Notwithstanding the morality (and legality) of the issue, here’s another reason why you shouldn’t do it – there’s a great chance you’ll be found out. If that happens, you can find yourself in serious legal jeopardy.

So just how easy is it to show that someone has stolen your code?

Typically the first step is for the (future) plaintiff to have their suspicions aroused. If half the engineering department leaves and starts up a company with a competing product, then it’s hardly surprising that your ex-employer will be suspicious. Of course suspicions aren’t grounds for a lawsuit. The plaintiff needs at least some evidence of your malfeasance. Now sometimes this can be done purely by the functionality / look and feel of a product. However in other cases it’s necessary for the plaintiff to get at your code’s binary image. You can make this very hard (and hence expensive) to do. However for your typical microprocessor, this step is surprisingly easy. Indeed there are any number of organizations around the world that are quite adept at extracting binary images from processors.  So what you may ask? I took the code, moved stuff around, used a different compiler and compiled it for a different processor, so good luck with showing that I used your code. Well the trouble with this, is that using tools such as Ida-Pro, it’s easy to count the number of functions in the code, and the arguments they take. These metrics are a remarkably good signature. [BTW, there are other metrics as well, but I really don’t want to give the whole game away]. Thus if the original code base and the stolen code base have a very similar function call signature, then there’s an excellent chance that the plaintiffs have enough evidence to file a lawsuit.

It’s at this point that you are really in trouble. As part of the lawsuit, the plaintiffs are allowed to engage in discovery.  In a case like this, it means quite simply that the court will require you to turn your source code over to an expert that has been retained by the plaintiffs (i.e. someone like yours truly). At this point, I can use any number of tools that are available for comparing code bases. Some of the tools are designed expressly for litigation purposes, while others are just some of the standard tools we use as part of our everyday work. Anyway, the point is this: these tools are really good at finding all sorts of obfuscations, including things such as:

  • Renaming variables, constants functions etc.
  • Changing function parameter orders
  • Replacing comments
  • Adding / deleting white space
  • Splitting / merging files

In many cases, they can even detect the plagiarism (theft) even if you have switched languages. In other words, if you have indeed stolen the source code, then the chances of  it not being conclusively proven at this stage are pretty slim. In short, life is about to get very unpleasant.

Having said the above, I like to think that the readers of this blog are not the type that would engage in source code theft. However I suspect that some of you have been tempted to go into business competing against your current employer. If this describes you, then what should you do to ensure that you don’t get hit with a lawsuit a year or two after starting your own business?  Well clearly the best bet is not to go into a competing business. However if you must do this, then get some legal advice (please don’t rely on what is written here – I’m just an engineer!) before you start. You will probably be advised to do a ‘clean room’ design, which in a nutshell will require you to demonstrate that the code in your competing product was designed from scratch, using nothing from your former employer. Be advised that even in these cases, if you adopt the same algorithms, then you may still be in trouble.




Simplify, then add lightness

Saturday, December 31st, 2011 Nigel Jones

As 2011 draws to a close I have reason to be thinking about things automotive. As part of my musings, I was reminded of the famous mantra espoused by the late Colin Chapman of Lotus Cars – ‘Simplify, then add lightness’. I have always liked this aphorism for its idea of ‘adding lightness’ rather than ‘subtracting weight’.

So what’s this got to do with embedded systems you ask? Well, when it comes to embedded systems programming, I think Chapman’s concept is spot on. While most people would probably agree that simplifying an embedded system is a good thing, I wonder though how many actively work so as to achieve this? In my case I regularly hold conversations with my clients which can be summarized as: Yes I can do that, but it comes at the expense of complexity – and complexity leads to bugs – and bugs are very expensive. I have found that clients are quite appreciative of this as it forces them to evaluate what is truly important about the product. Incidentally, when I first started in the consulting business, I thought that it was my job to give the client whatever it was they asked for. The fact that in doing so I was often giving them a serious disservice was quite a revelation when the penny finally dropped.

So what about the ‘adding lightness’ directive? Well clearly this is an interesting concept when it comes to firmware. I take it to mean that the code should be simple, easy to read, and fast in execution – in other words not unlike a Lotus sports car. If I have cause to re-read the code many months or years later and I find that the code has stood the test of time and is a pleasure to read, then I think I have achieved the required lightness. If by contrast the code is out of date and ugly to read then I know I have failed. In short it’s the difference between designing a Lotus Elan and an AMC Pacer.



Consulting as a leading economic indicator – update #2

Friday, February 25th, 2011 Nigel Jones

I have written before about consulting being a leading economic indicator. My hypothesis is that when companies need engineering help, but are unsure whether to take on employees, then they turn to consultants. Conversely when companies need to cut costs, the first to go are consultants and contractors. In short, consultants are the first to go in bad times and the first to be retained in good times. I posted an update in October 2010 where I reported that the consultants I know were seeing an increase in interest level – but not yet any real increase in actual work. So where are we 5 months later? Well my informal survey of other consultants confirms that the interest seen back in October has translated into a lot of work today. All the consultants I know are very busy; indeed their biggest problem seems to be managing demand. On this basis I’m quite confident that the embedded systems industry will see robust hiring here in the USA in the coming months. If you are looking to change jobs, it’s a good time to start dusting off the resume.

Evaluating embedded code

Sunday, June 20th, 2010 Nigel Jones

One of the interesting aspects of being an embedded systems consultant is that I get to look at a lot of code written by others. This can come about in a number of ways, but most commonly occurs when someone wants changes made to an existing code base and the original author(s) of the code are no longer available. When faced with a situation such as this, it is essential that I quickly get a sense for how maintainable the code is – and thus how difficult changes will be. As a result I have developed a few techniques to help me assess code maintainability which I thought I’d share with you.


After installing the code on my system, the first thing I do is run SourceMonitor over the code. SourceMonitor is a free utility that computes various metrics. The metrics and their values for a typical code base of mine are shown below.

Number of Files: 476

Lines of code: 139,013

Statements: 61,144

% branches: 6.3%

% comments: 41.7%

Functions: 2,509

Average statements / function: 11.8

Max Complexity: 158

Max depth: 9+

Average depth: 0.54

Average complexity: 2.38

Probably the only thing that needs explanation is ‘complexity’. The author of SourceMonitor is not computing the McCabe complexity index, but rather is computing complexity based upon Steve McConnel’s methodology. The details of the implementation aren’t particularly important to me, as I’m more interested in comparative values.

While SourceMonitor helps give me the big picture, it is nowhere near enough – and this is where it gets interesting.

Optimization Level

The next thing I look at are the optimization levels being used. These can be very revealing. For example if a high level of optimization is being used for the debug build then it might be indicative that a non-optimized build either will not fit into the available memory, or possibly that the code doesn’t run fast enough unless optimization is turned on. Either is indicative of a system that is probably going to be tough to maintain. Conversely if the release build doesn’t use full optimization then I take this to mean that the code probably doesn’t work when optimization is turned on. I have written about this in the past and consider this to be a major indicator of potential code quality problems.

C-V Qualifiers

Having looked at the optimization levels, I then perform a grep on the code base looking for the number of instances of ‘volatile‘ and ‘const‘. If the number of instances of volatile is zero (and it often is) and the optimization level is turned way down, then it’s almost certain that the author of the code didn’t understand volatile and that the code is riddled with potential problems. Whenever this happens, I get a sinking feeling because if the author didn’t understand volatile, then there is no chance that he had any appreciation for race conditions, priority inversion, non-atomic operations etc. In short, the author was a PC programmer.

The ‘const‘ count is less revelatory. If the author makes use of const then this is normally an indicator that they know their way around the compiler and understand the value of defensive programming. In short I take the use of const to be very encouraging. However, I can say that I have known some excellent embedded systems programmers who rarely used const, and thus its absence doesn’t fill me with the same despair as the absence of volatile.

Incidentally in my code base described above, there are 53 incidences of the use of ‘volatile’ (note that I have excluded compiler vendor supplied header files which define all the various hardware registers as volatile). There are also 771 incidences of the the use of const.

Static qualifiers

Regular readers of this blog will know I am a big fan of the ‘static‘ qualifier. Static not only makes for safer and more maintainable code, it also makes for faster code. In fact, IMHO the case for static is so overwhelming that I find its absence or infrequent use a strong indicator that the author of the code was an amateur. In my example code base, static appears 1484 times.

Case statements

Regular readers of this blog also know that I am not a big fan of the case statement. While it has its place, too often I see it used as a substitute for thought. Indeed I have observed a strong inverse correlation between programmer skill and frequency of use of the case statement. As a result, I will usually run a grep to see what the case statement frequency is. In my example code, a case statement occurs 683 times, or once every 90 statements.


All of the above ‘tests’ can be performed without compiling the code. In some cases I own the target compiler (or can download an evaluation copy), in which case I will of course attempt to compile the code. When I do this I’m looking for several things:

  1. An absence of compiler warnings / errors. Alan Bowens has written concisely and eloquently on this topic. The bottom line – compilation warnings in the release build are a major issue for me. Note that I’m more forgiving of compiler warnings in the debug build, since by its nature debug often ignores things such as inline commands, which can generate warnings on some compilers.
  2. The compilation speed. Massive files containing very large functions compile very slowly. They are also a bear to maintain.
  3. The final image size. This is relevant both in absolute terms (8K versus 128K versus 2M) and also in comparison to the available memory. Small images using a small percentage of the available memory are much easier to maintain than large images that nearly fill the available memory.


The final test that I perform only rarely is to Lint the code base. I do this rarely because quite frankly it takes a long time to configure PC-Lint. Thus only if I have already created a PC-Lint configuration file for the target compiler do I perform this step. Previously un-linted code will always generate thousands of warnings. However, what I’m looking for are the really serious warnings – uninitialized variables, indexing beyond the end of an array, possible null pointer dereferences etc. If any of these are present then I know the code base is in bad shape.

I can typically run the above tests on a code base in an hour or so. At the end of it I usually have a great idea of the overall code quality and how difficult it will be to modify. I would be very interested to hear from readers that are willing to perform the same tests on their code base and to publish the results. (Incidentally, I’m not trying to claim that my metrics are necessarily good – they are intended merely as a reference / discussion point).