Monthly Archives: October 2009

How good are your backups?

PICT0008We all know we need to do backups. But that is only half the story. Have you actually checked you can read them back if you need to? I have heard stories of people religiously backing up to mag tape every day for years, only to find out the tapes were corrupt and couldn’t be read back when needed.

I checked my backups recently to ensure I could read them back. Here is what I found out:

  • I was backing up my SVN repository on my Mac Mini to a single .tar.gz file which I then copied across onto a USB disk attached to a Windows box. The file had grown unnoticed to >4GB in size. But the USB disk was in FAT format, which has a maximum file size of 4GB. The the file was quietly being truncated to 4GB and I couldn’t uncompress the file when I tried with gunzip.
  • I was backing up from my Windows box to DVDs in ‘live’ format. These were not readable by my Mac mini, which would have been a problem if neither of my Windows boxes were working. I have now changed to ‘mastered’ format, which is readable by Windows and Mac.
  • I was backing up the .mdf file my OnTime bug tracking database. It turns out you can’t just swap one .mdf file for another and re-start OnTime, as you get an internal consistency error. I am now creating and backing up .bak files, which you can restore.

I have lots of redundancy in my backups – backing up in multiple formats (files, SVN repository and Acronis disk images) to multiple media (multiple machines, USB disks, DVD and online). So none of the above would have spelt disaster. But it does bring home the importance of testing your backups from time to time and of having multiple forms of backup. If you think backing up to a single USB disk is enough you should read this. If you are relying purely on an third party online backup service you should consider what would happen if they went bankrupt – not inconceivable in the current economic climate.

Are you relying on a single backup strategy? When was the last time you tested your backups?

Why I won’t be bothering with the Windows 7 logo program

not compatible with Windows 7Am I the only one being totally bombarded with ‘Give your application the green light’ and related emails from Microsoft and its minions? I must have had at least 30 so far. I took a few minutes to list my product in the Windows 7 compatibility guide (beware, cheesy audio). But that is all I intend to do.

I went to the trouble of getting the ‘works with Vista’ logo in 2007. The process was very broken:

  • The winqual and Partner websites give me “certified by unknown authority” warnings.
  • The Winqual website didn’t work at all in FireFox.
  • There was a complete lack of clear guidance about what you needed to do next at each stage.
  • The documentation was very poor, with broken links and much of it completely out of date (i.e. “more information will be available in 2006″, this was 2007).
  • It took me about an hour of rummaging around on the winqual site and an email to tech support to work out that you can ‘sign’ the legal documents online (you have to tick a permissions checkbox on a separate page and do a few refreshes).
  • You had to download a signing tool. It was in a zip file with a password. They didn’t tell you what the password was! Luckily I already had signcode.exe installed.

It was easily the most frustrating thing I have done in my career as a microISV. For that I ended up with an ugly ‘works with Vista’ logo (that probably just made customers think my software didn’t work on Windows XP), an entry in Windows marketplace (I already had one) and a Verisign authenticode certificate that I wasn’t allowed to use to sign my software.

Hopefully Microsoft have cleaned up their broken logo process since the Vista launch. But the benefits of the Windows 7 logo program seem slim:

  • a “Compatible with Windows 7″ logo (prettier than the ghastly “works with Vista” logo admittedly)
  • 30 Partner Points for use in the Microsoft Partner Program (I have no idea what I would want those for)
  • some PR templates (when did having a near identical press release to thousands of other companies become a benefit?)
  • priority Listing in the “Windows 7 Compatibility Center” (I doubt any of my customers know or care about this)
  • Windows built-in error reporting (I rolled my own, thanks)

Worst of all I would have to buy another overpriced Verisign certificate to authenticate myself to Microsoft winqual, even though I already have a perfectly valid authenticode certificate from Comodo.

I think I’ll pass.

A survey of ecommerce providers for software vendors

Overview

The choice of ecommerce provider is probably one of the more important ones you make as a software vendor. It isn’t too hard to compare providers by feature set or price. But what about other vital attributes, such as support, reliability, ease of set-up and how they treat your customers? It isn’t realistic to try every provider, so this major decision is often made on the basis of haphazard anecdotal evidence from forums. I created a survey in an attempt to gather some systematic data on the ecommerce providers most commonly used by small software vendors. I present the results below without fear or favour. Skip ahead to ‘Overall ranking’ if you are in a hurry.

Methodology

I posted a request for survey responses on this blog and on a few forums frequented by microISVs and small software companies. Any vendor of software (desktop or web based) not directly affiliated with an ecommerce provider was eligible to take part. Software vendors were invited to fill out a survey form on wufoo.com for each ecommerce provider they had used in the last 2 years. They had to supply their product URL and an email address from the same domain so that I could verify their identity. They also had to check a box proclaiming:

I am a software vendor and I have used this Ecommerce provider in the last 2 years. I have no commercial interest beyond being a customer. (If you have affiliate links to the Ecommerce provider, that isn’t a problem.)

They then had to reply to an automated email from wufoo to the email supplied confirming it was them that had completed the form. If they didn’t reply to the automated email I followed up with a few more emails. Although tedious for me, I felt this was an important safeguard to avoid any possibility of fraudulent entries. I also checked for duplicate entries, duplicate IP addresses and other suspicious patterns. The survey was open from the 5th to the 8th October. Any responses not validated by 10th October were removed from the data.

The data

202 survey responses were received from 166 different software vendors. 9 responses were rejected as I could not verify their identity (they didn’t respond to several emails). 1 response was rejected due to a possible conflict of interest raised by the software vendor (they had done paid work for one of the providers). This left 192 valid responses. I saw no evidence of any attempt to rig the results.

You can download the raw data. It has been stripped of any personal identifying information. Feel free to do your own analysis or check my results.

Providers

The survey listed 14 of the major ecommerce providers, plus an ‘other’ box for providers not listed. Valid responses were received for 25 different ecommerce providers, as shown below:

responses

Note that ‘e-junkie+PayPal/GoogleCheckout/2Checkout’ has been shortened to ‘e-junkie’ for brevity.

Questions

Below I show the average (mean) score per ecommerce provider by survey question. The providers are sorted by score. Providers with less than 3 responses weren’t considered statistically valid and are not shown here (see the raw data for all responses).

Features

“How do you rate the range of features offered, e.g. coupons, support for multiple currencies, CD shipping, affiliate tracking, handling of tax etc.”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

features

Ease of use

“How easy is their system to set-up, manage and modify?”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

ease_of_use

Reliability

“How reliable is their service? Does their server ever go down?”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

reliability

Support

“How good is their support? Do they respond in a timely manner? Are their staff helpful and knowledgeable?”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

support

Fraud protection

“How well do they protect you from chargebacks and false positives (i.e. valid cards declined)?”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

fraud_protection

Ethics

“Does this service disrespect you (e.g. by paying you late) or your customer (e.g. by spamming them, adding unwanted items into their cart or making hidden charges)?”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

ethics

Value for money

“How do you rate their service compared to the cost?”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

value_for_money

Future

“What is the probability you will still be using this service in 12 months time?”

5= “>95%”, 4= “>75%”, 3= “>50%”, 2= “>25%”, 1= “<25%”

futureThe average score and standard deviation for each question across all providers is shown below:

question_analysis

From the averages software vendors are most happy with reliability and least happy with ease of use. From the standard deviation the least variation is in fraud protection and the greatest variation is in support.

The correlation (R squared) between the likelihood of staying with a provider and the answers to the other 6 questions is shown below:

correlation

Perhaps providers should be concentrating more on ease of use and support to differentiate themselves from the competition.

Providers

Below I show the average (mean) score per question by ecommerce provider. The providers are shown in alphabetical order. The standard deviation is also shown to give an idea of how consistent the responses were (the larger the standard deviation the more variation there was in responses). Providers with less than 3 responses weren’t considered statistically valid and are not shown here (see the raw data for all responses).

avangate

bmt_micro

e-junkie

esellerate

fastspring

kagi

paypal

plimus

regnow

shareit

swreg

Overall ranking

The average (mean) score and overall ranking for providers with at least 3 responses is shown below.

overall

The chart below shows the score broken down by question (click to enlarge):

overall_detailed

The chart below compares the 4 top performers by question:

top_performers

Avangate, Fastspring, BMT Micro and e-junkie all did well. The difference between the Avangate and Fastspring score (approx 0.3%) is probably too small to be statistically significant, but the survey shows significant differences between the best and worst providers. SWREG trails in last place by quite a margin, managing to place last or second to last in an impressive 7 out of 8 questions. It is also noticeable that the providers owned by industry heavyweight Digital River fill 4 out of the bottom 5 places in the ranking. This rather begs the question of how they got to be the industry heavyweight in the first place.

Note that the ranking does not show who the ‘best’ ecommerce provider is, for the following reasons:

  • ‘Best’ depends on your requirements. All the questions have been equally weighted here. If you decide (for example) that good support should be weighted higher than ease of use you might come up with a quite different ranking.
  • The assignment of numerical values to responses (e.g. Excellent=5, Good=4 etc) was done for easier analysis, but is entirely arbitrary. Different values might have resulted in a different ranking.
  • We aren’t comparing like with like. Software vendors using ‘lightweight’ e-commerce providers such as PayPal or e-junkie might have lower expectations than software vendors using ‘fully featured’ e-commerce providers .
  • e-junkie, SWREG, BMT Micro and RegNow had respectively only 8, 7, 5 and 3 responses. They are therefore vulnerable to statistical fluctuations.

That said, the ranking does correlate fairly well with the many comments I see about ecommerce providers on various forums. I don’t think I would want to use any of the providers in the bottom half of the ranking.

Conclusion

While one shouldn’t take the overall ranking too seriously, it is clear that there are major differences in the performance of ecommerce providers in important areas other than pricing and features. I hope these results will allow software vendors (myself included) to make a better informed choice of ecommerce provider. Hopefully this will, in turn, improve ecommerce services overall by rewarding the good companies at the expense of the poorer performers. It would be interesting to run this survey in another year or two and find out what has changed. Thank you to everyone that took part.

Disclosure: I use e-junkie+PayPal/GoogleCheckout/2Checkout as my payment provider for my Perfect Table Plan software. I have an affiliate link to them in another article on this blog which brings me a few dollars a month. I have no other commercial relationship with any of the other ecommerce providers.

BMT Micro
e-junkie
eSellerate
Fastspring
Kagi
PayPal
Plimus
RegNow
ShareIt
SWREG

How good is your Ecommerce provider?

ecommerce surveyIt is important to choose the right Ecommerce provider for your business. A bad choice can have a significant impact on your sales and switching provider can be a major headache. But which one is the right one? It is easy enough to find out about prices and features, but what about the all-important intangibles such as support, ease of set-up and reliability? I hear a lot of good and bad reports about various vendors. I thought it was time for something a bit more comprehensive and systematic – a survey. That’s where you come in.

I hope this survey will provide a useful resource to software vendors looking for an Ecommerce provider and also force the under-performers to raise their game. But I need your help. So please click the link below and tell me what you think about your Ecommerce provider. Please note:

  • You must be a software vendor (web or desktop) and I need your email address and product website to prove this. You will have to reply to an automatic email after the survey to verify your identity. Without this it would be easy to rig the results. Your address and domain will remain confidential and won’t be used for any other purposes.
  • If you have used more than one Ecommerce provider in the last 2 years you can fill out a separate survey response for each one.
  • As there are quite a lot of Ecommerce providers I think I will need at least 100 responses to get a good data set. 200 would be better. So tell a friend.

** the survey is now closed **

results here