Posts Tagged ‘opensource’

Trends in Embedded Software Design

Wednesday, April 18th, 2012 Michael Barr

In many ways, the story of my career as an embedded software developer is intertwined with the history of the magazine Embedded Systems Design. When it was launched in 1988, under the original title Embedded Systems Programming (ESP), I was finishing high school. Like the vast majority of people at that time, I had never heard the term “embedded system” or thought much about the computers hidden away inside other kinds of products. Six years later I was a degreed electrical engineer who, like many EEs by that time in the mid-90′s, had a job designing embedded software rather than hardware. Shortly thereafter I discovered the magazine on a colleague’s desk, and became a subscriber and devotee.

The Early Days

In the early 1990s, as now, the specialized knowledge needed to write reliable embedded software was mostly not taught in universities. The only class I’d had in programming was in FORTRAN; I’d taught myself to program in assembly and C through a pair of hands-on labs that were, in hindsight, my only formal education in writing embedded software. It was on the job and from the pages of the magazine, then, that I first learned the practical skills of writing device drivers, porting and using operating systems, meeting real-time deadlines, implementing finite state machines, the pros and cons of languages other than C and assembly, remote debugging and JTAG, and so much more.

In that era, my work as a firmware developer involved daily interactions with Intel hex files, device programmers, tubes of EPROMs with mangled pins, UV erasers, mere kilobytes of memory, 8- and 16-bit processors, in-circuit emulators, and ROM monitors. Databooks were actual books; collectively, they took up whole bookshelves. I wrote and compiled my firmware programs on an HP-UX workstation on my desk, but then had to go downstairs to a lab to burn the chips, insert them into the prototype board, and test and debug via an attached ICE. I remember that on one especially daunting project eight miles separated my compiler and device programmer from the only instance of the target hardware; a single red LED and a dusty oscilloscope were the extent of my debugging toolbox.

Like you I had the Internet at my desk in the mid-90s, but it did not yet provide much useful or relevant information to my work other than via certain FTP sites (does anyone else remember FTPing into sunsite.unc.edu? or Gopher?). The rest was mostly blinking headlines and dancing hamster; and Amazon was merely the world’s biggest river. There was not yet an Embedded.com or EETimes.com. To learn about software and hardware best practices, I pursued an MSEE and CS classes at night and traveled to the Embedded Systems Conferences.

At the time, I wasn’t aware of any books about embedded programming. And every book that I had found on C started with “Hello, World”, only went up in abstraction from there, and ended without ever once addressing peripheral control, interrupt service routines, interfacing to assembly language routines, and operating systems (real-time or other). For reasons I couldn’t explain years later when Jack Ganssle asked me, I had the gumption to think I could write that missing book for embedded C programmers, got a contract from O’Reilly, and did–ending, rather than starting, mine with “Hello, World” (via an RS-232 port).

In 1998, a series of at least three twists of fate spanning four years found me taking a seat next to an empty chair at the speaker’s lunch at an Embedded Systems Conference. The chair’s occupant turned out to be Lindsey Vereen, who was then well into his term as the second editor-in-chief of the magazine. In addition to the book, I’d written an article or two for ESP by that time and Lindsey had been impressed with my ability to explain technical nuances. When he told me that he was looking for someone to serve as a technical editor, I didn’t realize it was the first step towards my role in that position.

Future Trends

Becoming and then staying involved with the magazine, first as technical editor and later as editor-in-chief and contributing editor, has been a highlight of my professional life. I had been a huge fan of ESP and of its many great columnists and other contributors in its first decade. And now, looking back, I believe my work helped make it an even more valuable forum for the exchange of key design ideas, best practices, and industry learning in its second decade. And, though I understand the move away from print towards online publishing and advertising, I am nonetheless saddened to see the magazine come to an end.

Reflecting back on these days long past reminds me that a lot truly has changed about embedded software design. Assembly language is used far less frequently today; C and C++ much more. EPROMs with their device programmers and UV erasers have been supplanted by flash memory and bootloaders. Bus widths and memory sizes have increased dramatically. Expensive in-circuit emulators and ROM monitors have morphed into inexpensive JTAG debug ports. ROM-DOS has been replaced with whatever Microsoft is branding embedded Windows this year. And open-source Linux has done so well that it has limited the growth of the RTOS industry as a whole–and become a piece of technology we all want to master if only for our resumes.

So what does the future hold? What will the everyday experiences of embedded programmers be like in 2020, 2030, or 2040? I see three big trends that will affect us all over those timeframes, each of which has already begun to unfold.

Trend 1: Volumes Finally Shift to 32-bit CPUs

My first prediction is that inexpensive, low-power, highly-integrated microcontrollers–as best exemplified by today’s ARM Cortex-M family–will bring 32-bit CPUs into even the highest volume application domains. The volumes of 8- and 16-bit CPUs will finally decline as these parts become truly obsolete.

Though you may be programming for a 32-bit processor already, it’s still true that 8- and 16-bit processors still drive CPU chip sales volumes. I’m referring, of course, to microcontrollers such as those based on 8051, PIC, and other instruction set architectures dating back 30-40 years. These older architectures remain popular today only because certain low-margin, high-volume applications of embedded processing demand squeezing every penny out of BOM cost.

The limitations of 8- and 16-bit architectures impact the embedded programmers in a number of ways. First, there are the awkward memory limitations resulting from limited address bus widths–and the memory banks, segmenting techniques, and other workarounds to going beyond those limitations. Second, these CPUs are much better at decision making than mathematics–they lack the ability to manipulate large integers efficiently and have no floating-point capability. Finally, these older processors frequently lack modern development tools, are unable to run larger Internet-enabled operating systems, such as Linux, and don’t feature the security and reliabiltiy protections afforded by an MMU.

There will, of course, always be many applications that are extremely cost-conscious, so my prediction is not that they will disappear completely, but that the overall price (including BOM cost as well as power consumption) of 32-bit micro controllers, with their improved instruction set architectures and transistor geometries, will win on price. That will put the necessary amount of computing power into the hands of some designers and make our work easier for all of us. It also helps programmers accomplish more in less time.

Trend 2: Complexity Forces Programmers Beyond C

My second prediction is that the days of the C programming language’s dominance in embedded systems are numbered.

Don’t get me wrong, C is a language I know and love. But, as you may know firsthand, C is simply not up to the task of building systems requiring over a million lines of code. Nonetheless, the demanded complexity of embedded software has been driving our systems towards more than a million lines of code. At this level of complexity, something has to give.

Additionally, our industry is facing a crisis: the average age of an embedded developer is rapidly increasing and C is generally not taught in universities anymore. Thus, even as the demand for embedded intelligence in every industry continues to increase, the population of skilled and experienced C programmers is on the decline. Something has to give on this front too.

But what alternative language can be used to build real-time software, manipulate hardware directly, and be quickly ported to numerous instruction set architectures? It’s not going to be C++ or Ada or Java, for sure–as those have already been tried and found lacking. A new programming language is probably not the answer either, across so many CPU families and with so many other languages already tried.

Thus I predict that tools that are able to reliably generate those millions of lines of C code automatically for us, based on system specifications, will ultimately take over. As an example of a current tool of this sort that could be part of the trend, I direct your attention to Miro Samek’s dandy open source Quantum Platform (QP) framework for event-driven programs and his (optional) free Quantum Modeler (QM) graphical modeling tool. You may not like the idea of auto-generated code today, but I guarantee that once you push a button to generate consistent and correct code from an already expressive statechart diagram, you will see the benefits of the overall structure and be ready to move up a level in programming efficiency.

I view C as a reasonable common output language for such tools (given that C can manipulate hardware registers directly and that every processor ever invented has a C compiler). Note that I do expect there to be continued demand for those of us with the skills and interest to fine tune the performance of the generated code or write device drivers to integrate it more closely to the hardware.

Trend 3: Connectivity Drives Importance of Security

We’re increasingly connecting embedded systems–to each other and to the Internet. You’ve heard the hype (e.g., “Internet of things” and “ubiquitous computing”) and you’ve probably already also put TCP/IP into one or more of your designs. But connectivity has a lot of implications that we are only starting to come to terms with. The most obvious of these is security.

A connected device cannot hide for long behind “security through obscurity” and, so, we must design security into our connected devices from the start. In my travels around our industry I’ve observed that the majority of embedded designers are largely unfamiliar with security. Sure some of you have read about encryption algorithms and know the names of a few. But mostly the embedded community is shooting in the dark as security designers, within organizations that aren’t of much help. And security is only as strong as the weakest link in the chain.

This situation must change. Just as Flash memory has supplanted UV-erasable EPROM, so too will over-the-net patches and upgrades take center stage as a download mechanism in coming years and decades. We must architect our systems first to be secure and then to accepted trusted downloads so that our products can keep up in the inevitable arms race against hackers and attackers.

And That’s a Wrap

Whatever the future holds, I am certain that embedded software development will remain an engaging and challenging career. And you’ll still find me writing about the field at http://embeddedgurus.com/barr-code and http://twitter.com/embeddedbarr.

Tools to Detect Software Copyright Infringement

Thursday, September 23rd, 2010 Michael Barr

An emerging class of tools makes it easy to automatically detect copying of copyrighted software source code, even if it came from one of the hundreds of thousands of open source packages.

I am presently providing litigation support in a case of alleged software copyright infringement.  In a nutshell, the plaintiff brought suit against the defendant for allegedly continuing to use plaintiff’s copyrighted software source code in defendant’s products after termination of a license agreement between the parties.  Fortunately, automated tools are making it easier than ever to quickly and inexpensively detect copying of software source code.

Some of the most powerful tools for doing direct comparisons between a pair of source code sets are from S.A.F.E. Their CodeMatch tool works by comparing each file of source code in the first set with every file of code in the second set.  Results are presented in a table that is sorted by the relative amount of matching code in the files.  And CodeMatch is clever enough to detect copying in which variable and function names and other details were subsequently changed; CodeMatch can even detect code that was copied from one programming language into another.  The only weakness of CodeMatch is that you have to have the source code for each product, which is not always possible early in litigation.

Other tools from S.A.F.E. provide additional help.  For example, BitMatch can compare a pair of executable binary programs or one party’s source code against another’s executable code.  It works by matching strings that appear in both programs.  Meanwhile, SourceDetective helps rule out that the two programs are only similar because they both borrowed from some third program—by automatically searching the Internet for hundreds or thousands of matching phrases.  CodeMatch, BitMatch, and SourceDetective are part of a suite of related tools called CodeSuite.  CodeSuite is a free download that runs on Microsoft Windows, with license keys sold based on the amount of code to be compared.

Of course, sometimes code may be copied from open source software.  Open source software is subject to so-called copyleft licenses, which are a special type of copyright that makes the source code open to the public.  Copyleft language is drafted to ensure that the source code for certain categories of derived work are also open to the public.  This creates problems for companies that wish to keep their source code private but also rely upon open source software.

Fortunately, there are also tools to detect the presence of part of all of an open source software package within a proprietary program.  I have used such tools from Black Duck Software and Protecode.  Both work similarly: each company maintains a database of hundreds of thousands of known open source packages against which the source code you provide is tested. Results are presented as a list of open source packages from which code may have been copied. This testing can be done entirely on a personal computer running Microsoft Windows, so that proprietary source code need not be sent outside a trusted network.  Both tools are generally licensed for an expected level of use on an annual basis.

Unfortunately, the precision of CodeMatch is lost in trying to cast such a broad net for potential copying.  The tools from BlackDuck and Protecode don’t actually compare your code against each and every of the millions of source code files in their database.  Instead, they reduce each file of your source code to a simpler representation of its structure and then compute a unique mathematical signature for that new file.  This signature is subsequently compared to a similar representation of the files in their database.  In plain English, this means that you get lots of false positives.  Some open source packages that weren’t actually copied usually turn up in the results list.

When searching for potential copying of open source code, I recommend searching the database from BlackDuck or Protecode first.  Then, to eliminate the false positives, a more thorough analysis should be performed by obtaining the listed open source packages and using CodeMatch to compare the proprietary code against them file-by-file.

With the help of tools like those mentioned here, it is possible to quickly ascertain whether source code copying has taken place.  Prior to the appearance of these tools, it was necessary for an expert in software development to manual perform dozens of searching and comparison steps.  This strategy can be used early in litigation with the benefit of dramatically reducing the cost of such analysis.  The same tools can also be employed proactively by companies seeking to reduce their risks of copyright infringement litigation.

Free as in, well, Free Software

Wednesday, September 27th, 2006 Michael Barr

There’s no such thing as free beer. But free software abounds. It seems that everywhere I look these days companies are offering their embedded operating systems and tools for free evaluation. Often, the price includes full access to the source code.

Examples just this week include the announcement that Quantum Leaps would make the source code for its previously proprietary QP-nano product available under GPL, that Micrium would release the source code for the TCP/IP stack it developed at great expense under a 45-day evaluation license, and that Hitachi‘s brand new Entier relational database could be downloaded for use in 30-day trials.

Given access to the source code of a complicated product such as an operating system, network stack, or relational database how many people pay? A restrictive license is perhaps a legal consideration, but if these guys can afford to give their source code out wily nily how do you feel about being the only schmuck to actually cough up dough? Does anyone buy embedded software components anymore?

Open Sores

Saturday, January 5th, 2002 Michael Barr

In the past two years, increasing numbers of embedded programmers have been getting to know Linux and other open source software packages intimately. What has primarily attracted this interest is the non-existent pricing structure. But some of the initial enthusiasm—particularly for Linux—seems to be fading.

Is the use of open source software as building blocks for embedded systems just a fad?
I’ve just found a couple of interesting insights about Linux buried within a recent survey of embedded developers by Evans Data Corporation. The survey asked a number of questions focused on Linux, and the results are cross-tabulated in interesting ways. One table, titled “Perceptions of Linux’ Biggest Technical Difficulties by Degree of Community Interaction,” presents data gleaned from a question asked of those considering and already using Linux to various degrees, sorted by their experience level. Developers who hadn’t actually done anything with Linux yet (about 84% of those surveyed) perceived its biggest technical hurdles to be “availability of device drivers” and “lack of board support packages.” However, developers with hands-on Linux experience including kernel modifications (about 6%) were most concerned about the “size” of the package.

You’d think that the size of the Linux code (which is measured in Megabytes), its worst-case interrupt latency and other performance characteristics, and RAM requirements (also Megabytes) would be the overriding concerns for embedded programmers. And yet the big issues that I hear everyone complain about are legalities surrounding open source licensing terms and fragmentation of the, widely distributed, code base. In reality, these latter are not big problems for embedded programmers—as those who’ve actually investigated Linux already know. It’s the memory and performance issues that really get in our way.

As the reality begins to overtake the hype, a consultant/author friend had this to say about the evolving market for his Linux services:
Two years ago I was pumped up on embedded Linux. You said it would pass; I thought you were crazy. Well… I just stopped work on my book. I only found two Linux clients and I ran out of money. Back to VxWorks to pay the bills—and get me out of debt for the time and effort I put into Linux.

Though there are certainly companies out there embedding Linux, the market isn’t growing as rapidly as most analysts predicted it would.