Successful Software

Have you ever noticed that you rarely (if ever) meet someone who admits to having below average driving skills? My Grandfather started driving a car before UK driving tests became compulsory in 1935. So he never had to take a driving test. This was lucky for him, because he was a terrible driver. He would get distracted and cross the line into oncoming traffic, veering back at the last second when his passenger started shouting. He claimed he had never been in accident, but I expect he would have seen quite a few if he had ever thought to look in his mirrors. Few people would accept a lift from him a second time. Even as a young boy, I realised that I was in mortal danger getting into his car. I would make an excuse and make my own way by bicycle. And yet, he considered himself a good driver. After having read the excellent book Bad science by Doctor, journalist and blogger Ben Goldacre, I think I know why.

The book contains this startling graph from the paper ‘Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments’ by Kruger and Dunning :

Graphics courtesy of explorativeapproach.com (click to enlarge)

The graph shows how good subjects were at logical reasoning as measured by a test (blue), how good they thought they were  (green) and how well they thought they did at the test (red). Apart from the top quartile, there was actually an inverse relationship between how skilled people thought they were and how skilled they actually were. The study also showed that the least competent individuals were also the least capable at recognizing the skill levels of others. This is the abstract from the paper:

People tend to hold overly favorable views of their abilities in many social and intellectual domains. The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it. Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd. Several analyses linked this miscalibration to deficits in metacognitive skill, or the capacity to distinguish accuracy from error. Paradoxically, improving the skills of participants, and thus increasing their metacognitive competence, helped them recognize the limitations of their abilities.

Or, more succinctly:

Ignorance more frequently begets confidence than does knowledge. Charles Darwin

The findings fit in with the concept of ‘unconsciously incompetent’ in Maslow’s four stages of learning:

1. Unconscious Incompetence:  The individual neither understands nor knows how to do something, nor recognizes the deficit, nor has a desire to address it.
2. Conscious Incompetence:  Though the individual does not understand or know how to do something, he or she does recognize the deficit, without yet addressing it.
3. Conscious Competence: The individual understands or knows how to do something. However, demonstrating the skill or knowledge requires a great deal of consciousness or concentration.
4. Unconscious Competence: The individual has had so much practice with a skill that it becomes “second nature” and can be performed easily (often without concentrating too deeply). He or she may or may not be able to teach it to others, depending upon how and when it was learned.

The Dunning-Kruger effect might explain why every crank and barstool scientist appears to think they understand global warming better than the world’s top climate scientists. I am also reminded of my own experiences learning ju-jitsu. After a year or two of training, having reached the exalted rank of green belt, I thought I was pretty good. It was only some years later, as a black belt, that I realised how much I still had to learn.

What has this got to do with software? Well, it might explain some of the very poor website design, GUI design, graphics and copywriting I see from time to time. The perpetrators may be sufficiently unskilled, that they don’t even realise how unskilled they really are. It is a thorny problem. Firstly you don’t realise you have the problem. Secondly, even if someone convinces you to delegate or outsource that type of work, you are unable to accurately assess the work of others. It is particularly worrying for one man software companies (such as myself) who have to perform or oversee a very wide range of  skills including: website design, user interface design, programming, testing, copyrighting, marketing, PR, documentation, support and systems admin.

By the same token,  many customers with poor IT skills might not have any insight into the extent of their deficit. Below are the results of a survey I did with some of my own customers a couple of years ago:

Note how skewed the results are and bear in mind that relatively few of my customers are IT professionals. Similarly skewed results were reported recently by Patrick McKenzie in his blog:

Surprisingly many of my customers self-evaluate as comfortable with computers.  50% were “very comfortable”, and 30% were “mostly comfortable”.  These numbers are, candidly speaking, not what I would have assigned on the basis of reading support requests for three years.

I don’t have any easy solutions for this problem. All you can do is:

• accept that you might not be the best judge of your own competence in all areas
• actively solicit feedback from your customers and your peers and listen carefully
• be a bit more tolerant of newb@aol.com when they blame your software for problems arising from their lack of basic IT knowledge
• console yourself that, whatever your IT shortcomings, at least you are a good driver