I have recently returned from  a short trip to the UK. I flew both ways on what appeared to be a relatively new Boeing 777 courtesy of  United Airlines. As is now common place on trans-Atlantic wide-body aircraft, my seat came with its own in-flight entertainment system. After I took my seat to fly to London, I was a little surprised to see that the in-flight entertainment system was suddenly rebooted. How did I know this? Well there was cute Linux penguin in the top left hand corner, plus I (and the rest of the plane) was treated to the always delightful task of watching hundreds of lines of startup script scrolling across the screen. After a minute or two the reboot came to an end and I was presented with the now standard touch screen user interface.This should have given me a hint that the in-flight entertainment system wasn’t the most stable of applications.

Now flights from the east coast of the USA to Europe tend to be night flights, and this flight was no exception. Being a seasoned traveler I have my trans-Atlantic night flight routine down pat. Get settled in, have something to eat, put on the noise cancelling headphones, select a classical music channel and then try and sleep for the rest of the flight. I followed my routine, and after selecting the appropriate music channel, I selected the volume control icon. This resulted in a pop-up window from which I adjusted the volume to something reasonable. I then made the fateful mistake. Rather than exiting the volume adjust window, I simply hit the ‘on/off’ button. This of course doesn’t turn the system off, it merely blanks the display. I then settled in for my nap. Several hours later, I woke up and wondered how far into the flight we were and thus decided to take a look at the real-time route map. Accordingly I turned the display ‘on’ and was rewarded with what I expected to see, namely the volume control screen. A photograph is shown below:

When I touched the Back button, nothing happened. Hmmm thought I, has my system died? Answer – no. I could still adjust the volume, and I could still mute and un-mute the audio. I could also turn the system ‘on’ and ‘off’. Clearly the problem was that there was no ‘back’ associated with the Back button. Needless to say, United Airlines wasn’t going to reboot the entire system so that I could experiment some more, plus I was tired. Anyway I resolved to experiment some more on the return flight.

Thus a week later I’m on a Boeing 777 with the same in flight entertainment system. I brought up the volume control page and did nothing. After about 30 seconds, the pop-up window auto cleared. This was the clue I needed. I thus brought up the volume control page again and immediately turned the unit ‘off’ and then a few seconds later ‘on’. When I did this the back button worked correctly. I repeated the exercise a few more times, and it always worked. I then repeated the exercise, but this time I waited longer than the screen timeout period. Voila – lockup. Clearly this was a case where there was a gaping hole in the state machine that was driving the user interface. At this point I found some interesting thoughts crossing my mind:

• Idiot. Now you can’t watch any movies.
• I bet the designer didn’t use a formal state machine tool such as visualSTATE, or QP
• Whenever there are two distinct ways of exiting a state (in this case, user action or the passage of time), life gets complicated
• Preserving state across a ‘power down’ is always difficult
• I hope the guy that wrote the in-flight entertainment  system had nothing to do with the flight control systems on the plane!

Anyway if you find yourself on a plane with an in-flight entertainment system in the near future, see if you too can crash the system – and let us know how you did it.

P.S. I woke up this morning to read that a computer ‘glitch’ effectively grounded United Airlines yesterday. See this for thoughts on United’s computer system.

One of the major differences between embedded systems engineers and the general public is that we tend to notice embedded systems a lot more – both when they do something very well, and also of course when they do things not so well. The latter happened to me recently when I was pursuing one of my hobbies, namely riding my bicycle. I’m quite a keen cyclist, in part because I happen to live in part of the world which has some terrific riding country. (To see what I mean, check out the photos of the ride in question. They were taken by my cycling (and skiing) buddy Bill – who I think you’ll agree is a very fine photographer. Click here to see more of his published work. If you hunt carefully, you’ll find some pictures of yours truly).

Anyway at about 25 miles into the ride, I noticed that my Incite bike computer was showing that I had ridden 34 miles. This struck me as unlikely, and so I asked Bill what mileage he had on his computer – 25 being the answer. I thus cycled through the computer screens, until I came to this one (photo courtesy of Bill Tan).

Apparently the computer thought that I had hit 132.6 mph at some point – and obviously sustained it for quite a while for me to gain about 9 miles. Now to understand how this could come about, you need to know a little about my bike computer. The computer consists of two parts, the User Interface (shown above) and the pick up. The pick up senses wheel rotation by a magnet passing close to what I assume is a Hall Effect Sensor. Now whereas many bike computers transfer the signal along a cable to the display, mine transmits it using an RF link – and this I suspect was the root cause of the problem. My guess is that at some point in the ride, I rode into an area of RF interference that the display interpreted as signals from the pickup. The firmware in the bike computer appears to have blithely accepted the RF data as valid and thus produced the ridiculous result shown here.

Now I have often been faced with this kind of problem – and the solution is not easy. However IMHO Incite really fell down on the job here. If I had been writing the code, I suspect that I’d have done the following:

1. Median filter the data to remove random outliers. (Incite may be doing this).
2. Sanity check the output of the median filter. If the answer is ‘impossible’ (like a human pedaling a bicycle at 132.6 mph), then reject the data and let the user know that something is amiss. Incite did neither of these things.

Rejecting the data is actually a little harder than it sounds. If you reject data, what do you replace it with? Common choices are:

1. Zero
2. The most recent valid data
3. The average of the last N readings.

Each of these has its place and is application dependent.

Letting the user know that something is amiss, is usually straight forward – flashing the erroneous value is the normal solution.

Anyway, the bottom line is that a wise embedded engineer always sanity checks the incoming (and outgoing) data. If you do you are less likely to end up as the subject of a blog posting.

A personal note

I apologize for the abysmal rate at which I have been posting. I moved house this spring, which coupled with me being extremely busy has resulted in there simply not being enough hours in the day. When that happens, something has to give. I hope to return to my normal blog posting rate in July.