Archive for the ‘productivity’ Category

My Embedded Toolbox: Source Code Whitespace Cleanup

Monday, August 7th, 2017 Miro Samek

In this installment of my “Embedded Toolbox” series, I would like to share with you the free source code cleanup utility called QClean for cleaning whitespace in your source files, header files, makefiles, linker scripts, etc.

You probably wonder why you might need such a utility? In fact, the common thinking is that compilers (C, C++, etc.) ignore whitespace anyway, so why bother? But, as a professional software developer you should not ignore whitespace, because it can cause all sorts of problems, some of them illustrated in the figure below:

qclean_dirty

  1. Trailing whitespace after the last printable character in line can cause bugs. For example, trailing whitespace after the C/C++ macro-continuation character ‘\’ can confuse the C pre-processor and can result in a program error, as indicated by the bug icons.
  2. Similarly, inconsistent use of End-Of-Line (EOL) convention can cause bugs. For example, mixing the DOS EOL Convention (0x0D,0x0A) with Unix EOL Convention (0x0A) can confuse the C pre-processor and can result in a program error, as indicated by the bug icons.
  3. Varying amount of trailing whitespace at the end of the lines plus inconsistent use of tabs and spaces can cause unnecessary churn in the version control system (VCS) in source files that otherwise should be identical. Sure, many VCSs allow you to “ignore whitespace”, but are files differing in size by as much as 20% really identical?
  4. Inconsistent use of tabs and spaces can lead to different rendering of the source code by different editors and printers.

Note: The problems caused by whitespace in the source code are particularly insidious, because you don’t see the culprit. By using an automated whitespace cleanup utility you can save yourself hours of frustration and significantly improve your code quality.

 


 QClean Source Code Cleanup Utility

QClean is a simple and blazingly fast command-line utility to automatically clean whitespace in your source code. QClean is deployed as natively compiled executable and is located in the QTools Collection (in the sub-directory  qtools/bin ). QClean is also available in portable source code and can be adapted and re-compiled on all desktop platforms (Windows, POSIX –Linux, MacOS).

Using QClean

Typically, you invoke QClean from a command-line prompt without any parameters. In that case, QClean will cleanup white space in the current directory and recursively in all its sub-directories.

Note: If you have added the qtools/bin/ directory to your PATH environment variable (see Installing QTools), you can run qclean directly from your terminal window.

qclean_run

As you can see in the screen shot above, QClean processes the files and prints out the names of the cleaned up files. Also, you get information as to what has been cleaned, for example, “Trail-WS” means that trailing whitespace has been cleaned up. Other possibilities are: “CR” (cleaned up DOS/Windows (CR) end-of-lines), “LF” (cleaned up Unix (LF) end-of-lines), and “Tabs” (replaced Tabs with spaces).

QClean Command-Line Parameters

QClean takes the following command-line parameters:

 PARAMETER  DEFAULT  COMMENT
[root-dir] . root directory to clean (relative or absolute)
OPTIONS
-h help (show help message and exit)
-q query only (no cleanup when -q present)
-r check also read-only files
-l[limit] 80 line length limit (not checked when -l absent)

QClean Features

QClean fixes the following whitespace problems:

  • removing of all trailing whitespace (see figure above 1)
  • applying consistent End-Of-Line convention (either Unix (LF) or DOS (CRLF) see figure above 2)
  • replacing Tabs with spaces (untabify, see figure above 2)
  • optionally, scan the source code for long lines exceeding the specified limit (-l option, default 80 characters per line).

Long Lines

QClean can optionally check the code for long lines of code that exceed a specified limit (80 characters by default) to reduce the need to either wrap the long lines (which destroys indentation), or the need to scroll the text horizontally. (All GUI usability guidelines universally agree that horizontal scrolling of text is always a bad idea.) In practice, the source code is very often copied-and-pasted and then modified, rather than created from scratch. For this style of editing, it’s very advantageous to see simultaneously and side-by-side both the original and the modified copy. Also, differencing the code is a routinely performed action of any VCS (Version Control System) whenever you check-in or merge the code. Limiting the line length allows to use the horizontal screen real estate much more efficiently for side-by-side-oriented text windows instead of much less convenient and error-prone top-to-bottom differencing.

QClean File Types

QClean applies the following rules for cleaning the whitespace depending on the file types:

 FILE TYPE  END-OF-LINE  TRAILING WS  TABS  LONG-LINES
.c Unix (LF) remove remove check
.h Unix (LF) remove remove check
.cpp Unix (LF) remove remove check
.hpp Unix (LF) remove remove check
.s Unix (LF) remove remove check
.asm Unix (LF) remove remove check
.lnt Unix (LF) remove remove check
.txt DOS (CR,LF) remove remove don’t check
.md DOS (CR,LF) remove remove don’t check
.bat DOS (CR,LF) remove remove don’t check
.ld Unix (LF) remove remove check
.tcl Unix (LF) remove remove check
.py Unix (LF) remove remove check
.java Unix (LF) remove remove check
Makefile Unix (LF) remove leave check
.mak Unix (LF) remove leave check
.html Unix (LF) remove remove don’t check
.htm Unix (LF) remove remove don’t check
.php Unix (LF) remove remove don’t check
.dox Unix (LF) remove remove don’t check
.m Unix (LF) remove remove check

The cleanup rules specified in the table above can be easily customized by editing the array l_fileTypes in the qclean/source/main.c file. Also, you can change the Tab sizeby modifying the TAB_SIZE constant (currently set to 4) as well as the default line-limit by modifying the LINE_LIMIT constant (currently set to 80) at the top of the the qclean/source/main.c file. Of course, after any such modification, you need to re-build the QClean executable and copy it into the qtools/bin directory.

Note: For best code portability, QClean enforces the consistent use of the specified End-Of-Line convention (typically Unix (LF)), regardless of the native EOL of the platform. The DOS/Windows EOL convention (CR,LF) is typically not applied because it causes compilation problems on Unix-like systems (Specifically, the C preprocessor doesn’t correctly parse the multi-line macros.) On the other hand, most DOS/Windows compilers seem to tolerate the Unix EOL convention without problems.

Summary

QClean is very simple to use (no parameters are needed in most cases) and is fast (it can easily cleanup hundreds of files per second). All this is designed so that you can use QClean frequently. In fact, the use of QClean after editing your code should become part of your basic hygiene–like washing hands after going to the bathroom.

My Embedded Toolbox: Programmer’s Calculator

Tuesday, June 27th, 2017 Miro Samek

Like any craftsman, I have accumulated quite a few tools during my embedded software development career. Some of them proved to me more useful than others. And these generally useful tools ended up in my Embedded Toolbox. In this blog, I’d like to share some of my tools with you. Today, I’d like to start with my cross-platform Programmer’s Calculator called QCalc.

qcalc

I’m sure that you already have your favorite calculator online or on your smartphone. But can your calculator accept complete expressions in the C-syntax, which you can cut-and-paste directly to and from your embedded code? How many buttons do you need to push to see your result in decimal, hex and binary? Well, QCalc can do this with less hassle than anything else I’ve seen out there. I begin with describing QCalc features and then I tell you how to download and launch it.

QCalc Features

Expressions in C-Syntax

The most important feature of QCalc is that it accepts expressions in the C-syntax – with the same operands and precedence rules as in the C or C++ source code. Among others, the expressions can contain all bit-wise operators (<<, >>, |, &, ^, ~) as well as mixed decimal, hexadecimal and even binary constants. QCalc is also a powerful floating-point scientific calculator and supports all mathematical functions (sin(), cos(), tan(), exp(), ln(), …). Some examples of acceptable expressions are:

((0xBEEF << 16) | 1280) & ~0xFF – binary operators, mixed hex and decimal numbers
($1011 << 24) | (1280 >> 8) ^ 0xFFF0 – mixed binary, dec and hex numbers
(1234 % 55) + 4321/33 – remainder, integer division
pow(sin($pi),2) + pow(cos($pi),2) – scientific floating-point calculations, pi-constant
($0111 & $GPIO_EXTIPINSELL_EXTIPINSEL0_MASK) << ($GPIO_EXTIPINSELL_EXTIPINSEL1_SHIFT * 12) – (see user-defined variables)

NOTE: QCalc internally uses the Tcl command expr to evaluate the expressions. Please refer to the documentation of the Tcl expr command for more details of supported syntax and features.

Automatic conversion to hexadecimal and binary

If the result of expression evaluation is integer (as opposed to floating point), QCalc automatically displays the result in hexadecimal and binary formats (see QCalc GUI). For better readability the hex display shows a comma between the two 16-bit half-words (e.g., 0xDEAD,BEEF). Similarly, the binary output shows a comma between the four 8-bit bytes (e.g., 0b11011110,10101101,10111110,11101111).

Binary constants

As the extension to the C-syntax, QCalc supports binary numbers in the range from 0-15 (0b0000-0b1111). These binary constants are represented as $0000$0001$0010,…, $1110, and $1111 and can be mixed into expressions. Here are a few examples of such expressions:

($0110 << 14) & 0xDEADBEEF
($0010 | $1000) * 123

History of inputs

QCalc remembers the history of up to 8 most recently entered expressions. You can recall and navigate the history of previously entered expressions by pressing the Up / Down keys.

The $ans variable

QCalc stores the result of the last computation in the $ans variable (note the dollar sign $ in front of the variable name). Here are some examples of expressions with the $ans variable:

1/$ans – find the inverse of the last computation
log($ans)/log(2) – find log-base-2 of the last computation

User variables

QCalc allows you also to define any number of your own user variables. To set a variable, you simply type the expression =alpha in the user input field. This will define the variable alpha and assign it the value of the last computation ($ans). Subsequently, you can use your alpha variable in expressions by typing $alpha (note the dollar sign $ in front of the variable name). Here is example of defining and using variable $GPIO_BASE:

0xE000E000 – set some value into $ans
=GPIO_BASE – define user variable GPIO_BASE and set it to $ans
$GPIO_BASE + 0x400 – use the variable $GPIO_BASE in an expression

Note: The names of user variables are case-sensitive.

Error handling

Expressions that you enter into QCalc might have all kinds of errors: syntax errors, computation errors (e.g., division by zero), undefined variable errors, etc. In all these cases, QCalc responds with the Error message and the explanation of the error:

qcalc_err

Downloading and Launching QCalc

QCalc is included in the open source QTools Collection, which you cad freely download from SourceForge or GitHub. Once you install QTools, QCalc is located in the sub-directory qtools/bin/ and consists of a single file qcalc.tcl. To launch QCalc, you need to open this file with the wish Tk interpreter.

NOTE: The wish Tk interpreter is included in the QTools Collection for Windows and is also pre-installed in most Linux distributions.

You use QCalc by typing (or pasting) an expression in the user input field and pressing the Enter key to evaluate the expression. You can conveniently edit any expression already inside the user input field, and you can recall the previous expressions by means of the Up/Down keys. You can also resize the QCalc window to see more or less of the input field.

QCalc on Windows

The wish Tk interpreter is conveniently provided in the same qtools/bin/ directory as the qcalc.tcl script. The directory contains also a shortcut qcalc, which you can copy to your desktop.

qcalc_lnk

QCalc on Linux

Most Linux distributions contain the Tk interpreter, which you can use to launch QCalc. You can do this either from a terminal, by typin wish $QTOOLS/qcalc.tcl & or by creating a shortcut to wish with the command-line argument $QTOOLS/qcalc.tcl.

qcalc_linux

Dual Targeting and Agile Prototyping of Embedded Software on Windows

Friday, April 12th, 2013 Miro Samek

When developing embedded code for devices with non-trivial user interfaces, it often pays off to build a prototype (virtual prototype) of the embedded system of a PC. The strategy is called “dual targeting”, because you develop software on one machine (e.g., Windows PC) and run it on a deeply embedded target, as well as on the PC. Dual targeting is the main strategy for avoiding the “target system bottleneck” in the agile embedded software development, popularized in the book “Test-Driven Development for Embedded C” by James Grenning.

Avoiding Target Hardware Bottleneck with Dual Targeting

Please note that dual targeting does not mean that the embedded device has anything to do with the PC. Neither it means that the simulation must be cycle-exact with the embedded target CPU.

Dual targeting simply means that from day one, your embedded code (typically in C) is designed to run on at least two platforms: the final target hardware and your PC. All you really need for this is two C compilers: one for the PC and another for the embedded device.

However, the dual targeting strategy does require a specific way of designing the embedded software such that any target hardware dependencies are handled through a well-defined interface often called the Board Support Package (BSP). This interface has at least two implementations: one for the actual target and one for the PC, for example running Windows. With such interface in place, the bulk of the embedded code can remain completely unaware which BSP implementation it is linked to and so it can be developed quickly on the PC, but can also run on the target hardware without any changes.

While some embedded programmers can view dual targeting as a self-inflicted burden, the more experienced developers generally agree that paying attention to the boundaries between software and hardware is actually beneficial, because it results in more modular, more portable, and more maintainable software with much longer useful lifetime. The investment in dual targeting has also an immediate payback in the vastly accelerated compile-run-debug cycle, which is much faster and more productive on the powerful PC compared to much slower, recourse-constrained deeply embedded target with limited visibility into the running code.

Agile Rapid Prototyping of Embedded Software with Dual Targeting

Dual targeting can have many different objectives. For example, in the test-driven development (TDD) of embedded software, the objective is to build relatively concise unit tests and execute them on the desktop as console-type applications. The main challenge is management of the inter-module dependencies and flexibility of tests, but the overall architecture of the final product is of lesser concerns, as the unit tests are executed in isolation using special test harnesses.

However, dual targeting can also be used for (rapid) prototyping and simulating the whole embedded devices on the PC, not just executing unit tests. In this case, the objective is to build a possibly complete prototype of the embedded device as a GUI-type application. This approach is particularly interesting for embedded systems with non-trivial user interfaces, such as: home appliances, office equipment, thermostats, medical devices, industrial controllers, etc. As it turns out, significant percentage of the code embedded in all those devices is devoted to the user interface and can be, or even should be, developed on the desktop.

QWIN GUI Toolkit

When developing embedded code for devices with non-trivial user interfaces, one often runs into the problem of representing the embedded front panels as GUI elements on the PC. The problem is so common, that I’m really surprised that my internet search couldn’t uncover any simple C-only interface to the basic elements, such as LCDs, buttons, and LEDs. I’ve posted questions on StackOverflow, and other such forums, but again, I got recommendations for .NET, C#, VisualBasic, and many expensive proprietary tools, none of which provided an easy, direct binding to C. My objective is not really that complicated, yet it seems that every embedded developer has to re-invent this wheel over and over again.

qwin_ani

 

 

So, to help embedded developers interested in prototyping embedded devices on Windows, I have created a QWIN GUI Toolkit” and have posted on SourceForge (as part of the Qtools collection) under the permissive MIT open source license. This toolkit relies only on the raw Win32 API in C and currently provides the following elements:

  • Graphic display for an efficient, pixel-addressable displays such as graphical LCDs, OLEDs, etc. with full 24-bit color.
  • Segment display for segmented display such as segment LCDs, and segment LEDs with generic, custom bitmaps for the segments.
  • Owner-drawn buttons with custom “depressed” and “released” bitmaps and capable of generating separate events when depressed and when released.

The toolkit comes with an example and an App Note, showing how to handle input from the owner-drawn buttons, regular buttons, keyboard, and the mouse. You can also view a 1-minute YouTube video “Flyn ‘n’ Shoot game on windows” that shows a virtual embedded board running a game.

Regarding the size and complexity of the “QWIN GUI Toolkit“, the implementation of the aforementioned GUI elements takes only about 250 lines of C. The example with all sources of input and a lot of comments amounts to some 300 lines of C. The toolkit has been tested with the free MinGW compiler, the free Visual C++ Express 2013, and the free ResEdit resource editor.

Enjoy!

The Best Christmas Present for a Nerd

Wednesday, December 5th, 2012 Miro Samek

Christmas is right around the corner and if you wonder about the presents, I have just an idea for you. No, it is not the new iPad, Galaxy S3 phone, or any of the new “ultrabooks”. In fact, this is exactly the opposite. My present idea is to boost your productivity in creating “content”, not merely consuming it.

And when it comes to creating anything with a computer, you need a big screen–the bigger the better. In fact, I’d recommend that you get yourself two new monitors. And don’t think small. How about two 27″, 1920x1080p full HD, LED-lit panes? You can get those for under $300 each, so a pair will still cost you less than a new iPad.

I got such a setup a few months ago, and now I’m absolutely convinced that this has been the best investment in my productivity–better than a faster CPU or a solid-state disk. I really can’t benefit from my machine being faster–that’s not what wastes my time. But I sure can use more screen, to read the documentation two pages at a time, and to see a complete IDE or a modeling tool on the other screen (modeling tools absolutely love big screens!).

The picture of my desk shows my setup. I have two 27″ HP 2711x 1080p monitors connected to an HP dv6 laptop. One monitor is connected via the HDMI cable and the other via the analog VGI cable. I don’t see any degradation in image quality on the VGA-driven monitor.

Dual Monitors

As you can see in the picture, I’ve placed my monitors on 6″ stands above my desk ($25 each). This is actually quite important, because too many people place their screens too low for comfortable work. (Using a laptop without a stand and additional keyboard is absolutely the worst!)

So, here it is: my Christmas present idea for a nerd. Write a letter to Santa about it, and maybe he will shove it down your chimney? (Only if you are good, that is!)