Archive for the ‘Uncategorized’ Category

Condition Based Maintenance For Automotive Systems

Saturday, June 11th, 2011 Mike Ficco

Modern cars have dozens of embedded systems – from adjusting the air/fuel mixture to preventing the brakes from locking – it seems nearly everything in the car is controller by embedded software.

This folks, is a revolution.  For a hundred years the automotive industry struggled to beat mechanical complexity into submission.  As progress was gradually made the demands on drivers became progressively less.  Early drivers had to be tinkerers with a toolkit and mechanical aptitude.  Today blissfully ignorant people get in their car, turn the key, and drive away.

Aside –

I once had a dog that would return from a summer walk, sit in front of an air conditioner floor vent, and periodically paw it.  Evidently he was rewarded often enough for his pawing that he came to believe he could control, or at least influence, the air conditioner gods.  Of course there was a random delay before the cool breeze started.  One could only wonder if his primitive cocker spaniel brain attributed this to the air conditioner gods being busy or perhaps angry with him.

 

There are times when all of us are cocker spaniels (but maybe not as cute).  In a sense, we are all cocker spaniel brained when confronted with befuddling new technology.  Indeed, Arthur C. Clarke pointed out “Any sufficiently advanced technology is indistinguishable from magic”.

To distill my thinking, we have a ubiquitous device – the car, of historic complexity, recently imbued with sophisticated embedded artificial intelligence.  Few who drive these cars understand the magic by which they operate.  Fewer still understand the limitations of that magic.  Worse, we cannot ignore the marketing and corporate agendas being applied to this unholy muddle.

For example, the people who want to sell you more oil tell you to change it every 3,000 miles while the people who want to sell you another car say an oil change every 7,500 or even 15,000 miles is fine.  Hummmmm.

Now we have all the ingredients for a real mess:

  • Complex and at times unproven technology
  • Multiple competing corporate agendas
  • Large population of uninformed users
  • Deep seated (and unwarranted) faith in magic
  • An abundance of cocker spaniel brains

 

A friend of mine has a fairly new car that displays a wrench icon on the dashboard when it is time for maintenance.  In the old days, a conscientious person would look at their odometer and make a service appointment at certain well-known mileage milestones.  15,000 and 30,000 miles are two examples of this traditional approach.  More recently, manufacturers have been making forays into Condition Based Maintenance (CBM).  This is cool stuff.  The idea behind CBM is that the need for maintenance depends on how hard you use something.  Embedded software in my friend’s car monitors sensors and calculates when to light up the wrench.  This is really cool until the cocker spaniels get involved.

Someone offered to change my friend’s oil around 9 or 10,000 miles.  Specific details are a little hazy.  Nobody is sure how much oil was used.  Nobody is sure if any computations or sensors were reset.  At around 15,000 miles my friend contacted the dealership to make a service appointment.  The service center representative asked if the wrench was visible on the dashboard.  It was not and the representative said not to bring the car in until the wrench was visible.  Questioned further, the representative said the wrench could not be confused by any out of cycle oil changes.  It knew when service was needed and don’t worry about bringing the car in until the wrench appears.

Note:  subsequent events and discussions with this dealership resulted in a complete retraction of this.  Evidently there is some sort of mileage based maintenance schedule.  It’s not my car and I’m not really clear on the details.  If you have a wrench on your dash – you should become knowledgeable about this.

Fast forward a couple thousand miles.  Still no wrench, but the oil warning light started coming on – first on hills, then most of the time.  Finally my friend decided to check the oil.  SURPRISE!  The engine oil was more than a quart low.  DOUBLE SURPRISE!  Hot transmission fluid barely touched the bottom of the dipstick.  We are not really sure what the wrench monitors, but clearly it doesn’t monitor fluid levels.  We’ve since learned it also does not monitor clogged air filters.  What exactly does the wrench monitor and why was it confused?  Good questions.

Now, remember – there is no real agreement among knowledgeable individuals and corporations on when you should change your oil.  3K, 5K, 7.5K, 15K…  Genuine oil change estimates are all over the map.  Given this, why would you:

  1. spend a penny in development
  2. add complexity to the manufacturing process
  3. confuse customers and service personnel

to tell someone to change their oil at 7,735 miles instead of 7,500?  This smells badly of marketing, not engineering.

Conditioned Based Maintenance is cool stuff and an excellent field for further research.  Unfortunately, its current use in cars seems badly mired in corporate politics and marketing rather than science.

 

Microsoft After Bill

Wednesday, May 25th, 2011 Mike Ficco

In 1998 I was working at a company that had tens of thousands of customers.  Like most companies, they dreamed of having tens of millions of customers.  The most significant impediment to their goal of mass-market acceptance was the futuristic application of their primary product.

In 1998, the Internet was still a baby, new PCs were made from the Pentium II, and less than half of them had CD readers.  It was this year, this company, and these circumstances that revealed to me the secret of Microsoft’s early success.

My company’s software had grown too big to fit on a single floppy.  It required three (3).  In 1998 Internet delivery of the software wasn’t even considered by my boss.  Considered, but rejected by the CEO, was delivery on a CD.  He didn’t want to lose access to over half the PCs by requiring a CD reader.  To put this in perspective, the company had less than 50,000 customers and the CEO decided that access to 20 or 30 or 40 million potential customer was just not enough.  He demanded a bigger pool.

The unfortunate consequence of his decision was the delivery of this futuristic product was being dumbed-down to retain compatibility with old PCs.  Here was a product, most likely to be attractive to “early adopters”, but it was being crippled by the antiquated image of having to repeatedly swap floppies to install it.

It was during this time that I started thinking about how Microsoft did things.  Bill Gates seemed to not care about under-powered and feature-poor PCs.  So what if the customer’s current PC was too weak to provide a good user experience, or perhaps too anemic to work at all with his new software?  Unlike most corporate leaders, Bill saw the future and didn’t wait for the world to catch up.

His approach, and it worked very well, was to provide desirable features and functionality attractive to the early adopters.  They in turn went out and bought new PCs so they could get maximum enjoyment from the new software.  In the process Bill as the CEO, from memory and very brief research, led Microsoft to something like a 10x stock price gain from when it went public in 1986 until he stepped down as CEO in 2000.  At that time the Microsoft stock price was about $45.  Here we are in 2011 and the stock is about $25.

This isn’t to malign the leadership that followed Bill – only to encourage them to do better.  Yes, Bill was good but without him Microsoft has needlessly become a follower.  In a sense, they worry they may lose customers by requiring a CD reader.  They’ve come to believe others are more innovative.  As a result they look to acquire web search technology, cloud anything, Skype, etc.  I don’t remember Microsoft creating any new market or ground-breaking technology since Bill stepped down.  Worse, and now we get to the reason for this blog, they are doing just plain stupid things.

Example 1:  Windows Explorer search

Have you tried searching for anything with the Vista or Windows 7 versions of Windows Explorer?  I first encountered this nonsense a few years ago on Vista.  Windows XP search was just fine.  You give it a full or partial file name and tell it what to look for inside the files that match – simple, easy, and it worked.  On Vista, there didn’t seem to be a way to tell the search to look inside the files.  After poking for quite a while and web searching I figured out how to force Explorer to look inside files.  Unfortunately, I never figured out how to specify file names so some searches became very painfully long as the search looked inside every file no matter how big or irrelevant.

Amazingly, they didn’t fix this in Windows 7.  Last week I was doing a search for a file but I just couldn’t find it.  I assumed I just remembered the content incorrectly.  Later, I found the file for which I was searching – and the content was as I remembered it.

Huh???

I went back and searched again.  Knowing where to look, Explorer could not find the file.  This file was standard ASCII text (in fact it was Python code).  I could see the text in Notepad and I could see the text in Eclipse.  Hummmmm?  Maybe I have to turn on Windows indexing?  So I tried that and it corrupted my Subversion project files and I had to reinstall the entire project.  Grrrr!!!

Summary

Hey Microsoft guys.  I’ve been using computers for over 30 years.  I’ve used more operating systems and file systems than you ever heard of – from Intel’s RMX, Univac, DEC’s RSX, Unix, Linux, DOS, Mac, and EVERY version of Windows ever made…

AND, after HOURS of playing…

I can’t figure out how to use the new Windows Explorer search.  This is a bad, BAD product.  Fix it!  And while you are doing that, put BACK the ability to tell the search file names to look in so I can help narrow the search.

Example 2:  The Windows 7 calculator.

What the HELL were you thinking?  Some time ago I was working on a record structure that had 16 bit binary time stamps.  Using the Windows 7 calculator in hex mode, I was converting the binary number to decimal seconds, then dividing by 60 to get the minutes.  After several records I was surprised they were all evenly dividing by 60.  After several more I became doubtful and actually started to look at the numbers instead of blindly typing.  Imagine my surprise when I discovered that the Microsoft calculator didn’t work!  In what crazy universe does it make since to force the “programmer” view of a decimal calculator to ignore decimals?  You do know that sometimes programmers use these things called floating point numbers, right?  You know, programmers don’t always use integers, right?

To do my binary seconds to minutes conversion, and see the result was not an even multiple of 60, I had to save each value and toggle to the “standard” view to do the division – then toggle back to the programmer view for each record.

Have you guys ever tried to use this?

BAD – BAD – BAD!!!

Fix it!!!

 

Infrared Photovoltaics Could Solve Energy and Climate

Thursday, March 31st, 2011 Mike Ficco

It may not be possible to overhype new infrared photovoltaic technology. It’s basically a solar cell powered by excess heat. Further developments promise to power cars and factories by cooling the planet.

Albert Einstein was first to describe the photoelectric effect, in 1907.  He was awarded the Nobel Prize for this work in 1923.  Briefly, the photoelectric effect occurs when a photon smacks into a substance and frees an electron.  When this happens regularly, an electric current is produced.  All solar cells are based on this phenomenon.  For many years visionaries have projected solar cells as the clean energy source of the future.  It looks like they may be right, but not in the way they expected.

Energy from Heat

Working together, Katzumi Suzuki of the Nipon Engineering Institute and Shrinavas Patel of the Engineering Foundation of Bombay reported in the Journal of Thermodynamic Physics that they created a successful experiment in which they lowered the photon energy needed to create the photoelectric effect to under one electron volt.  Such a low energy corresponds to a photoelectric “threshold frequency” in the infrared part of the spectrum.  In practical terms this means that solar cells made with their patented proprietary process are capable of producing electricity from infrared energy (i.e., heat).

Katzumi and Shrinavas report that today they can only achieve about 11% efficiency, but they hope to boost that to perhaps 18% within the next decade (their paper calculates a theoretical limit of 21.7%).  They are working to manufacture and sell one meter wide rolls of thin, flexible solar cell material of various widths and lengths.  No price has been quoted.

The amount of electricity generated is non-linear with temperature and, with the existing process, generation of electricity cannot be achieved at temperatures below -10 degrees Fahrenheit.  The Journal of Thermodynamic Physics noted that one square meter of material generated in the dark (i.e., no visible light) about 15 Watts at the freezing point and about 60 Watts at room temperature.  This means that a shirt made from this material could power a smartphone indefinitely from your own body heat.  A car covered in this material could drive for nearly 600 miles in an Arizona summer night.

Global Cooling

Perhaps the most important part of this discovery is its potential application in the field of climate change.  There are hints this technology could be used to cool global warming by transferring the surplus heat into charged batteries. Additional details can be found in the just published journal article.

What’s Wrong With Home Security?

Tuesday, March 15th, 2011 Mike Ficco

After living in my house for 22 years my illusion of security was shattered.  Three high school kids created a mini-crime spree, breaking into something like a dozen houses in my generally good neighborhood.  With squad cars roaming the streets and police helicopters overhead the group broke into their third house that day – mine.  They were caught but made a general mess of my place and a number of missing items were never recovered.  OK – fine…  Time to close the barn door after the horse escaped.  I need a security system.

Quick research taught me that, something like cell phone hardware and the phone company that provides the service, there is the security hardware and, separately, there is the company that provides the monitoring service.  I called one of the security companies and discussed my needs with an extremely well spoken woman who suggested I schedule an installation.  She offered that I was lucky to have called because they were in the middle of a major promotion.  She waived installation fees and gave me several free sensors.  She introduced me to her manager and made sure I was getting all the discounts available.  Cool, I thought, and was very impressed.

A few days later the installation technician showed up.  He removed his shoes before entering my house – another example of quality employees and good training, I thought.  Things took a negative turn, however, when I understood some equipment needed to be installed at the point where my phone line enters the house.  That was very inconvenient since doing so would require removing a cabinet and cutting into drywall.  The technician explained that the module needed to be installed at the point where the line enters the house and would not work correctly if simply connect to one of my many phone jacks.  Well, said I, that would have been good to know before we scheduled this appointment…

The technician pressed on and suggested an upgraded glass break detector.  He said the one I was to receive for free was not very reliable.  Finally, he suggested we upgrade to the GSM module so we would not need the home phone line to report a problem.  Using the GSM module would circumvent the need to use the home phone line – but unfortunately would add $10 per month in monitoring fees.  I decided to cancel the installation but the technician would hear none of that.  He insisted on putting me down for another appointment to “give me a chance to think about it”.

A couple of days later I called the company to cancel the new appointment.  What a change.  I was magically transported to a used car lot where I found myself apparently talking to a used car salesman – or at least it seemed that way.  They didn’t want to hear about canceling the appointment.  Wasn’t I aware of the crime statistics?  Didn’t I want my family to be safe?  I could have the GSM hardware for free (but NO discount on the monitoring).  On and on this continued and became difficult.  Eventually, I successfully cancelled the appointment.

What was I to do now?  I wanted a security system but didn’t want my wall damaged and didn’t want to pay extra fees.  I thought about the problem…

 

WAIT!!

Isn’t a home security system simply an embedded system with a remote connection?  Haven’t I worked for years on embedded systems that communicate via the Internet?  Don’t I have a perfectly good wireless LAN in my house?  Why can’t the security system just connect to my wireless LAN and handle all communication needs through the Internet?  I walked around my house and made a list of what I actually wanted and started calling security companies.

Oh my!  The world of security companies does not deal well with a customer that specifies what they want.  The business model of most of the companies seems based on some very simple principles:

  • Sell fear
  • Sell the 24 hour monitoring service
  • Avoid discussing details

Maybe I just had some bad luck, but I ran into very deceptive statements and practices.  I eventually DID get a security system and that salesman and the installation technician were very professional.  However, my brief experience in this area tells me the home security industry is well behind the technology curve and desperately needs to be dragged into the future.

The home monitoring companies have an extraordinarily profitable business model and, from a technology perspective, are years behind.  It seems to me the days of this business model are numbered.

 

How Utility Outages SHOULD Be Handled

Monday, February 21st, 2011 Mike Ficco

Earlier (How to Reduce Electric Utility Outages), I promised to provide a short engineering specification of how a utility might better inform the public of repair progress during an outage.  What I describe following will not only inform the public but could also help solidify good management practices within the utility.  Improvement in management practices seems likely to translate directly into better customer service through more rapid repairs.

Management Experience

As an experienced engineering manager I know you can’t always predict how long work will take.  However, my direct engineering experience has been that predictions become MUCH more reliable if the work is broken down into discrete and well-defined steps.  Even more important is to then arrange the steps into a sequence that builds on intermediate results and optimizes usage of the available resources.  In short, success comes more readily with a complete and detailed plan.

My experience has also been that making the detailed plan available for review, critique, and comment often results in an even better and more highly optimized plan.  Such review allows for the inclusion of missed steps and for correction of operational sequences that had not been fully considered.

It is without question that ad hoc direction of maintenance crews is far less efficient than having a comprehensive multi-step plan with a well-defined series of steps and an estimate of the time to complete each of these steps.

Therefore:

The utility must have such a plan.  If they have no such plan – they should and must be required to produce one for each significant outage.  Managers must either be trained or replaced until such an obviously needed basic component of good service is instinctively created and used for every major outage.  Note: Inexperienced managers sometimes argue against taking the time to produce a detailed plan.  They may say it is only delaying the start of repairs.  Wrong.  It has been proven over and over that, for major work, a good plan more than repays the time invested in producing it.

Good Management Practices:

  • It is good management and a demonstration of foresight to produce a couple of plans in advance of the emergency.
  • When the emergency occurs, it should only take minutes to select a relevant preexisting plan and “tweak” it for the current circumstances.
  • There is no reason the plan produced in the first half hour has to be the final plan.  Don’t be afraid to enhance it as the emergency progresses and more facts are learned.
  • Make use of the experience gained during each emergency and adjust the preexisting plans to be even better for the next emergency.

Since it is the job of the Public Utility Commission to oversee the utilities, they would be negligent if they did not insist on reviewing the preexisting plans.  They should also insist on participating in post-emergency plan reviews.

It is incontrovertible that a plan must exist to guide repairs for every major outage.  The question becomes how should this plan be made available to the public.  I believe a spreadsheet or Gantt chart would be highly inappropriate.  Instead, I proposed a graphic web page.

Requirements:

  1. The basic web page shall be a map of the region.
  2. It shall be possible to zoom the map from a high-level view of the entire region to individual street level.
  3. The utility grid shall be superimposed on the map.
  4. The utility grid shall be color coded as follows:
    • Green – Represents the portions of the utility grid known to be working correctly.
    • Gray – Indicates sections of the grid that have an unknown state.
    • Yellow – These sections are not functioning correctly but are currently under repair.
    • Orange – These sections are not functioning correctly and are next in line for repair.
    • Red – Represents the sections of the utility grid not working and not scheduled for repair in the immediate future.
  5. The current time of day shall be presented.
  6. The time of day the page data was last updated shall be presented.
  7. The page data shall be updated AT LEAST once an hour (update every 15 minutes is preferred).
  8. The average time in minutes that trucks/crews have been on their current assignments shall be presented.
  9. The number of currently working trucks/crews shall be presented.
  10. The number of trucks/crews on break or pending assignment shall be presented.
  11. In a major outage, a number of trucks/crews will likely be requested from neighboring jurisdictions.  Each group of requested trucks/crews shall be treated as a discrete block.  For each such block:
    • The source shall be named and the number of trucks/crews requested shall be indicated.
    • The expected arrival time shall be indicated along with the number of hours since the request was issued.
  12. For all the blocks in #11, the total number of all trucks/crews that have been requested from neighboring jurisdictions but have not yet arrived shall be presented.
  13. It shall be possible for the public to provide feedback on the emergency plan and its implementation.  This feedback shall be archived and made available to the Public Utilities Commission upon demand.

Epitaph

Over my two blogs on public utilities I’ve described a three-step process for improving the reliability and accountability of the system:

  1. Stop blaming your outages on Acts of God and start doing regular preventive maintenance and infrastructure improvements.  If you claim you have already been doing so, clearly your efforts have been inadequate and need to be improved.
  2. Prepare emergency plans and post these for review and comment.
  3. Create a regularly updated (perhaps as often as every 15 minutes) web site that shows the current state of repairs.  This is certainly needed during an emergency – but why not do it every day?

Maybe we can finally stop hearing promises and actually have utility company executives earn their bonuses not by “saving money” but by providing the reliable service implied by their social contract with the community.