Successful Software

Overview

The choice of ecommerce provider is probably one of the more important ones you make as a software vendor. It isn’t too hard to compare providers by feature set or price. But what about other vital attributes, such as support, reliability, ease of set-up and how they treat your customers? It isn’t realistic to try every provider, so this major decision is often made on the basis of haphazard anecdotal evidence from forums. I created a survey in an attempt to gather some systematic data on the ecommerce providers most commonly used by small software vendors. I present the results below without fear or favour. Skip ahead to ‘Overall ranking’ if you are in a hurry.

Methodology

I posted a request for survey responses on this blog and on a few forums frequented by microISVs and small software companies. Any vendor of software (desktop or web based) not directly affiliated with an ecommerce provider was eligible to take part. Software vendors were invited to fill out a survey form on wufoo.com for each ecommerce provider they had used in the last 2 years. They had to supply their product URL and an email address from the same domain so that I could verify their identity. They also had to check a box proclaiming:

I am a software vendor and I have used this Ecommerce provider in the last 2 years. I have no commercial interest beyond being a customer. (If you have affiliate links to the Ecommerce provider, that isn’t a problem.)

They then had to reply to an automated email from wufoo to the email supplied confirming it was them that had completed the form. If they didn’t reply to the automated email I followed up with a few more emails. Although tedious for me, I felt this was an important safeguard to avoid any possibility of fraudulent entries. I also checked for duplicate entries, duplicate IP addresses and other suspicious patterns. The survey was open from the 5th to the 8th October. Any responses not validated by 10th October were removed from the data.

The data

202 survey responses were received from 166 different software vendors. 9 responses were rejected as I could not verify their identity (they didn’t respond to several emails). 1 response was rejected due to a possible conflict of interest raised by the software vendor (they had done paid work for one of the providers). This left 192 valid responses. I saw no evidence of any attempt to rig the results.

You can download the raw data. It has been stripped of any personal identifying information. Feel free to do your own analysis or check my results.

Providers

The survey listed 14 of the major ecommerce providers, plus an ‘other’ box for providers not listed. Valid responses were received for 25 different ecommerce providers, as shown below:

Note that ‘e-junkie+PayPal/GoogleCheckout/2Checkout’ has been shortened to ‘e-junkie’ for brevity.

Questions

Below I show the average (mean) score per ecommerce provider by survey question. The providers are sorted by score. Providers with less than 3 responses weren’t considered statistically valid and are not shown here (see the raw data for all responses).

Features

“How do you rate the range of features offered, e.g. coupons, support for multiple currencies, CD shipping, affiliate tracking, handling of tax etc.”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

Ease of use

“How easy is their system to set-up, manage and modify?”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

Reliability

“How reliable is their service? Does their server ever go down?”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

Support

“How good is their support? Do they respond in a timely manner? Are their staff helpful and knowledgeable?”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

Fraud protection

“How well do they protect you from chargebacks and false positives (i.e. valid cards declined)?”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

Ethics

“Does this service disrespect you (e.g. by paying you late) or your customer (e.g. by spamming them, adding unwanted items into their cart or making hidden charges)?”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

Value for money

“How do you rate their service compared to the cost?”

5=”Excellent”, 4=”Good”, 3=”Satisfactory”, 2= “Unsatisfactory”, 1=”Dismal”

Future

“What is the probability you will still be using this service in 12 months time?”

5= “>95%”, 4= “>75%”, 3= “>50%”, 2= “>25%”, 1= “<25%”

The average score and standard deviation for each question across all providers is shown below:

From the averages software vendors are most happy with reliability and least happy with ease of use. From the standard deviation the least variation is in fraud protection and the greatest variation is in support.

The correlation (R squared) between the likelihood of staying with a provider and the answers to the other 6 questions is shown below:

Perhaps providers should be concentrating more on ease of use and support to differentiate themselves from the competition.

Providers

Below I show the average (mean) score per question by ecommerce provider. The providers are shown in alphabetical order. The standard deviation is also shown to give an idea of how consistent the responses were (the larger the standard deviation the more variation there was in responses). Providers with less than 3 responses weren’t considered statistically valid and are not shown here (see the raw data for all responses).

Overall ranking

The average (mean) score and overall ranking for providers with at least 3 responses is shown below.

The chart below shows the score broken down by question (click to enlarge):

The chart below compares the 4 top performers by question:

Avangate, Fastspring, BMT Micro and e-junkie all did well. The difference between the Avangate and Fastspring score (approx 0.3%) is probably too small to be statistically significant, but the survey shows significant differences between the best and worst providers. SWREG trails in last place by quite a margin, managing to place last or second to last in an impressive 7 out of 8 questions. It is also noticeable that the providers owned by industry heavyweight Digital River fill 4 out of the bottom 5 places in the ranking. This rather begs the question of how they got to be the industry heavyweight in the first place.

Note that the ranking does not show who the ‘best’ ecommerce provider is, for the following reasons:

• ‘Best’ depends on your requirements. All the questions have been equally weighted here. If you decide (for example) that good support should be weighted higher than ease of use you might come up with a quite different ranking.
• The assignment of numerical values to responses (e.g. Excellent=5, Good=4 etc) was done for easier analysis, but is entirely arbitrary. Different values might have resulted in a different ranking.
• We aren’t comparing like with like. Software vendors using ‘lightweight’ e-commerce providers such as PayPal or e-junkie might have lower expectations than software vendors using ‘fully featured’ e-commerce providers .
• e-junkie, SWREG, BMT Micro and RegNow had respectively only 8, 7, 5 and 3 responses. They are therefore vulnerable to statistical fluctuations.

That said, the ranking does correlate fairly well with the many comments I see about ecommerce providers on various forums. I don’t think I would want to use any of the providers in the bottom half of the ranking.

Conclusion

While one shouldn’t take the overall ranking too seriously, it is clear that there are major differences in the performance of ecommerce providers in important areas other than pricing and features. I hope these results will allow software vendors (myself included) to make a better informed choice of ecommerce provider. Hopefully this will, in turn, improve ecommerce services overall by rewarding the good companies at the expense of the poorer performers. It would be interesting to run this survey in another year or two and find out what has changed. Thank you to everyone that took part.

Disclosure: I use e-junkie+PayPal/GoogleCheckout/2Checkout as my payment provider for my Perfect Table Plan software. I have an affiliate link to them in another article on this blog which brings me a few dollars a month. I have no other commercial relationship with any of the other ecommerce providers.