Monthly Archives: January 2008

Seeing your software through your customers’ eyes

usabilityWe all like to think that our software is easy to use. But is it really? How do you know? Have you ever watched anyone use it? When I asked this questions to a room full of developers last year I was surprised at how many hadn’t.

Other people don’t see the world the way you do. Their weltanschauung (view on the world) is influenced by their culture, education, expectations, age, gender and many other factors. Below is a copy of a card I received for my birthday a few weeks ago (click for a larger image) which I think illustrates the gulf between how developers and their customers see the world rather well.

birthday_card.jpg

If your customers are also developers the difference in backgrounds may not be so large. But the difference in how they see your software and how you see it is still huge. You have been working on your software for months or years. You know everything worth knowing about it down to the last checkbox and command line argument. But your potential customer is probably going to download it and play with it for just a few minutes, or a few hours if you are lucky, before they decide if it is the right tool for the job. If they aren’t convinced, your competitors are only a few clicks away. To maximise your chances of making a sale you need to see your software afresh through your customer’s eyes. You can get some useful feedback from support emails, but the best way to improve the ease of use of your software is to watch other people using it. This is usually known as usability testing.

The basic idea of usability testing is that you take someone with a similar background to your target audience, who hasn’t seen your software before and ask them to perform a typical series of tasks. Ideally they should try to speak out loud what they are thinking to give you more insight into their thought processes. You then watch what they do. Critically, you do not assist them, no matter how irresistible the urge. The results can be quite surprising and highly revealing. Usability testing can be very fancy with one way mirrors, video cameras etc, but that really isn’t necessary to get most of the benefits. There is a good description of how to carry out usability tests in Krug’s excellent book Don’t make me think: a common sense guide to web usability. Most of his advice is equally applicable to testing desktop applications.

The main problems with usability testing are logistical. You need to find the right test subjects and arrange the time and location for testing. You also need to decide how you are going to induce them to give up an hour of their time. Worst of all, once you have used someone they are ‘tainted’ and can’t be used again (except perhaps to test changes in the new versions). It’s a hassle. Or at least it was. Much of this hassle is now taken care of for you by new web-based service www.usertesting.com .

The idea behind usertesting.com is very simple. You buy a number of tests for your website and specify your website url, the tasks you want carried out and the demographics (e.g. preferred age, gender and expertise of testers). Testers are then selected for you and carry out the testing. Once tests have been completed a flash audio+video recording of the session and a brief written report is uploaded for you. Finally you rate the testers on a 5-star scale. Presumably testers who score well will get more work in future. Ideally you should re-run your usability testing after any changes to verify that they are an improvement. I don’t know if usertesting.com allows for the fact that you probably won’t want the same tester a second time for the same project.

I paid $57 for 3 tests on perfecttableplan.com. I was happy with the tests, which pointed out a number of areas I can improve on. There was a problem which meant one of the tests still hadn’t been completed 4 days later. I emailed support and they sorted this out in a timely fashion. It is a new service and they are still ironing out a few glitches. Given the low costs and the 30 day money back guarantee I think it is definitely worth a try. It won’t take many extra conversions to repay your investment. usertesting.com is probably more useful to those of us selling to the wider consumer market. If you are selling to specialised niches (e.g. developers, actuaries, llama breeders) they might have difficulty finding suitable testers.

Unfortunately usertesting.com is currently only available for website usability testing. When I emailed them to suggest they extend the service to desktop apps they told me that it this might be a possibility if there was sufficient interest. I will be first in-line if such a service becomes available. Until then I am left with the hassle of organising my own usability tests. It occurs to me that I could do this remotely using a service such as copilot.com (now free at weekends)+Skype. This might be a good workaround for the fact that my office isn’t really big enough for two people (especially if they don’t know me very well!). It would also allow me to do testing with customers outside the UK, e.g. professional wedding planners in the USA. If I do try this I will report back on how I get on.

Nokia to acquire Trolltech (makers of Qt)

trolltechTelecoms giant Nokia are to acquire Trolltech, the company behind leading cross-platform toolkit Qt. As a fan and long time user of Qt this makes me a little nervous. I hope it will lead to more investment in Qt and lower maintenance fees. The opposite could happen of course. It isn’t unknown for a large company to buy a good product and then ruin it (the purchase of Purify by Rational springs to mind). At least I have the full source code for Qt if it all goes to hell in a handbasket. I learnt long ago (the hard way) that you should never depend on a third party libraries if you don’t have the source.

Facebook don’t need no steenkin’ software ads

facebook.gifI don’t really ‘get’ Facebook. Maybe that is because I am 42 years old and I am not supposed to. But I do get the advertising potential. Facebook have cunningly extracted detailed demographic data from their customers and are using this to offer highly targeted advertising to businesses. For example, Facebook currently has 32,080 females, aged 25-40, with a college education, in the USA who are engaged. These are the sort of people I would love to advertise my seating planner software to. But Facebook doesn’t want my money. A quick read through the Facebook advertising guidelines reveals:

No ad is permitted to contain or link, whether directly or indirectly, to a site that contains software downloads, freeware, or shareware.

This might possibly be due to worries about malware, but that seems to be covered by other clauses. Maybe they just want to keep their customers on the Facebook site, so they can click more ads? But, aren’t most URLs going to indirectly link to sites containing software downloads if you keep following links?

Google Adwords: improving your ads

google adwordsOne of the keys to success in Google Adwords (and other pay per click services) is to write good ad copy. This isn’t easy as the ads have a very restrictive format, reminiscent of a haiku:

  • 25 character title
  • 2×35 character description lines
  • 35 character display url

Whats more, there are all sorts of rules about punctuation, capitalisation, trademarks etc. You will soon find out about these when you write ads. Most transgressions are flagged immediately by Google algorithms, others are picked up within a few days by Google staff (what a fun job that must be).

Google determines the order in which ads appear in their results using a secret algorithm based on how much you bid, how frequently people click your ads and possibly other factors, such as how long people spend on your site after clicking. Nobody really knows apart from Google, and they aren’t saying. The higher your click frequency, generally the higher your ad will appear. The higher your ad appears in the results, generally the more clicks you will get. So writing relevant ads is very important. This means that each adgroup should have a tightly clustered group of keywords and the ads should be closely targeted to those keywords.

There is no point paying for clicks from people who aren’t interested in your product, so you need to clearly describe what you are offering in the few words available. For example you might want to have a title “anti-virus software” instead of “anti-virus products” to ensure you aren’t paying for useless clicks from people looking for anti-viral drugs (setting “drugs” as a negative keyword would also help here).

I have separate campaigns for separate geographic areas. Each campaign contains the same keywords in the same adgroups, but with potentially different bid prices and ads. This allows me to customise the bid prices and ads for the different geographic areas. For example I can quote the £ price in a UK ad and the $ price in a US ad. Having separate campaigns for separate geographic areas is a hassle, but it is manageable, especially using Google Adwords editor.

Writing landing pages specific to each adgroup can also help to increase your conversion rate. It is worth noting that the ad destination url doesn’t have to match the display url. For example you could have a destination url of “http://www.myapp.com/html/landingpage1.html?ad_id=123″ and a display url of “www.myapp.com/freetrial”.

Obviously what makes for good ad copy varies hugely with your market. Here are some things to try:

  • a call to action (e.g. “try it now!”)
  • adding/removing the price
  • different capitalisation and punctuation
  • keyword insertion (much beloved of EBay)
  • changing the destination url

But, as always, the only way to find out what really works is testing. Google have made this pretty easy with support for conversion tracking and detailed reporting. I run at least 2 ads in each adgroup and usually more. Over time I continually kill-off under-performing ads and try new ones. Often the new ads will be created by slight variations on successful ads (e.g. changing punctuation or a word) or splicing two successful ads together (e.g. the title from one and the body from another). This evolutionary approach (familiar to anyone that has written a genetic algorithm) gradually increases the ‘fitness’ of the ads. But you need to decide how to measure this fitness. Often it is obvious that one ad is performing better than another. But sometimes it can be harder to make a judgment. If you have an ad with a 5% click-through rate (CTR) and 0.5% conversion rate is this better than an ad with a 1% click-through rate and a 2% conversion rate? One might think so ( 5*0.5 > 1*2 ) but this is not necessarily the case. I think the key measure of how good an ad is comes from how much it earns you for each impression your keywords get.

I measure the fitness by a simple metric ‘profit per thousand impressions’ (PPKI) where, for a given time period:

PPKI = ( ( V * N ) – C ) / ( I / 1000 )

V = value of a conversion (e.g. product sale price)

N = number of conversions (e.g. product sales) from the ad

C = total cost of clicks for the ad

I = impressions for the ad

Say your product sells for $30. Over a given period you have 2 ads in the same adgroup that each get 40k impressions and clicks cost  an average of $0.10 per click.

  • ad1 has a CTR of 5%, a conversion rate of 0.5% and gets 10 conversions, which gives PPKI=$2.5 per thousand impressions
  • ad2 has a CTR of 1%, a conversion ratio of 2% and gets 8 conversions, which gives PPKI=$5 per thousand impressions

So ad2 made, on average, twice the profit per impression despite the lower number of conversions. Given this data I would replace ad1 with a new ad. Either a completely new ad or a variant of ad2.

PPKI has the advantage of being quantitative and simple to calculate. You can just export your Google Adwords ‘Ad Performance’ reports to Excel and add a PPKI column. Some points to bear in mind:

  • Selling software isn’t free. You may want to subtract the cost of support, CD printing & postage, ecommerce fees, VAT etc from the sale price to give a more accurate figure for the conversion value.
  • PPKI doesn’t take account of the mysterious subtleties of Google’s ‘quality score’. For example an ad with low CTR and high conversion rate might conceivably have a good PPKI but a poor quality score. This could result in further decreases in CTR over time (as the average position of the ad drops) and rises in minimum bid prices for keywords.
  • PPKI is a simple metric I have dreamt up, I have no idea if anyone else uses it. But I believe it is a better metric than cost per conversion, or any of the other standard Google metrics.

To ensure that all your ads get shown evenly select ‘Rotate: Show ads more evenly’ in your Adwords campaign settings. If you leave it at the default ‘ Optimize: Show better-performing ads more often’ Google will choose which ads show most often. Given a choice between showing the ads that make you most money and the ads which make Google most money, which do you think Google will choose?

Text ads aren’t the only type of ads. Google also offer other types of ads, including image and video ads. I experimented with image ads a few years ago, but they got very few impressions and didn’t seem worth the effort at the time. I may experiment with video ads in the future.

The effectiveness of ads can vary greatly. Back in mid-December I challenged some Business Of Software forum regulars to ‘pimp my adwords’ with a friendly competition to see who could write the best new ads for my own Google Adwords marketing. The intention was to inject some fresh ‘genes’ into my ad population while providing the participants with some useful feedback on what worked best. Although it is early days, the results have already proved interesting (click the image for an enlargement):

adwords ad results

The graph above shows the CTR v conversion ratio of 2 adgroups, each running in both USA and UK campaigns. Each blue point is an ad. The ads, keywords and bid prices for each ad group are very similar in each country (any prices in the ads reflect the local currency for the campaign). Points to note:

  • There were enough clicks for the CTR to be statistically significant, but not for the conversion rate (yet).
  • The CTRs vary considerably within the same campaign+adgroup. Often by factor of more than 3.
  • Adgroup 1 performs much better in the USA than in the UK. The opposite is true for adgroup 2.
  • Adgroup 1 for the USA shows an inverse correlation between CTR and conversion rate. I often find this is the case – more specific ads mean lower CTR but higher conversion rates and higher profits.

‘Pimp my adwords’ will continue for a few more months before I declare a winner. I will be reporting back on the results in more detail and announcing the winner in a future post. Stay tuned.

Oryx Digital is diversifying

BBC microI first became interested in programming in about 1978, at the age of 12. I can recall the exact moment. I was in a classroom at The Royal Hospital School watching a very basic demo (written in BASIC) of a ball bouncing around a screen on an RM-380Z. Actually it wasn’t a ball, it was a single pixel. But the screen resolution was so low it was easy enough to see from the back of the classroom. Computers with floppy drives were rather expensive for schools in 1978, but some pupils from the school had won it in a competition. I was intrigued – how did it work? The teacher giving the demonstration (Mr Albert) encouraged my early interest and a few years later my grandmother was generous enough to buy our family an Acorn BBC B computer. My future path was set.

30 years later, including 22 years as a professional software developer, I am still fascinated by software. Experience showed me programming skills were necessary, but far from sufficient, to produce successful commercial software. So my interests have grown from programming to include the whole nascent discipline of software engineering. I have also become increasingly interested in the effective marketing of software. Many developers recoil with horror from marketing, but I want my software to make money and be used by lots of people. This requires good marketing as well as a good product. In my experience talented software marketers are even harder to find than talented software developers, so I have learned as much as I can about marketing software. It is actually quite a challenging and creative field.

3 years ago I set up my own one-man company, Oryx Digital, to create software products and offer consulting services to other software companies. Since then I have been extremely busy developing and marketing my product, PerfectTablePlan, which has gone from strength to strength. I released PerfectTablePlan v3.1.1 for Windows and Mac OS X a few days ago. I am very pleased with this new version, which has over 50 improvements and new features. The response from customers has been very favourable and the software appears to be very stable – no automatic crash reports (yet). It has grown far beyond my original ideas and now weighs in at around 100K lines of C++ and 200 pages of user documentation. In my (biased) opinion it is way ahead of any of it’s competitors.

Although PerfectTablePlan remains my main focus, I feel now is a good time to diversify a little. So I am now making myself available a few days a month for consultancy to other software companies, large and small. Do you need a new perspective on your product development and marketing? Perhaps I can help?

Meanwhile I have already started thinking about PerfectTablePlan v4. No rest for the wicked…

Mobile Internet access

3-mobile-broadband.pngI try to check my sales and support emails at least twice a day, every day. I managed this 362 days in 2007 (I took a break for Christmas day and 2 days I was in Germany at a conference). But providing this level of service can prove to be a problem for a one-man software company when it comes to taking holidays. Last year I restricted holidays to places with broadband Internet access. But finding child-friendly accommodation with broadband access proved to be quite a headache.

I have considered getting a Blackberry, but I really need something that can run my application to do proper support.

After some dithering I have now finally got mobile Internet access for my laptop through 3 Networks at £10/month. This provides 1GB per month of free data in the UK. You can get a higher data allowance with a more expensive contract, or a pay-as-you-go contract. But 1GB/month will hopefully be sufficient for my needs. Data costs outside the UK are a frightening £6/MB, so I will probably have to look for alternative arrangements if I take a holiday abroad. Vodaphone offer contracts with more reasonable roaming rates, but the contracts are much more expensive (£25 – £99/month).

Installation of the USB mobile modem and software was very easy – it took me about 5 minutes from opening the packaging to being connected. Only time will tell how good the coverage and service are. Watch this space.

Having mobile Internet access could also be a useful back-up if I lose my landline broadband connection. This is quite reassuring after several website outages and a failed harddisk in the last couple of weeks.