Tag Archives: shipping

30 tips for creating great software releases

If a film director is only as good as their last film, then I guess a software developer is only as good as their last software release. In more than 30 years of writing software professionally I have shipped my fair share of releases. For the last 13 years I have been shipping software as a solo developer. Here are a few things I have learned along the way. Some of them are specific to downloadable software, but some of them apply equally to SaaS products.

Use a version control system

I occasionally hear about software developers who don’t use a version control system. Instead they usually create some sort of janky system using dated copies of source folders or zip files. This send shivers down my spine. Don’t be that guy. A version control system should be an essential part of every professional software developer’s tool kit. It matters less which version control system you use. All the cool kids now use distributed version control systems, such as git. But I find that Subversion is fine for my requirements.

Tag each release in version control

This makes it easy to go back and compare any two releases. A bug appeared in the printing between v1.1.1 and v1.1.2? Go back and diff the source files related to printing and review all the changes.

Store your release binaries in version control

I store every binary I ship to customers in my version control system. Many people will tell you that you should only store the source in version control. Then you can use this to regenerate the binaries if you need to. This was sound advice back in the day when harddisks were small, networks were slow, version control systems were clunky (SourceSafe!) and developer environments didn’t change very frequently. But I don’t think it is valid advice now. Harddisks are as cheap as chips, networks are much faster and online updates mean your SDK, compiler or some other element of your toolchain is likely to be updated between your releases, making it impossible for you to recreate an identical release binary later on.

‘Test what you ship, ship what you test’

In an analog system (such as a bridge) a tiny change in the system will usually only cause a small change in the behaviour. In a discrete system (such as software) a change to a single bit can make the difference between a solid release and a showstopper bug. The Mariner I rocket was destroyed by a single missing hyphen in the code. So test the binaries that you plan to ship to the customer. And if you change a single bit in the release, re-test it. You probably don’t need to run all the tests again. But you certainly can’t assume that a small change won’t cause a big problem.

This issue often manifests itself when the developers test the debug version of their executable and then ship the release version. They then find that the two have different behaviour, e.g. due to a compiler optimization, different memory layout or code inside an ASSERT.

Make each executable individually identifiable

As a corollary of the above, you need to be able to uniquely identify each executable. I do this by having a timestamp visible in the ‘About’ box (you can use __DATE__ and __TIME__ macros in C++ for this) and ensure that I rebuild this source file for every release.

Diff your release with the previous one

Do a quick diff of your new release files versus the previous ones. Have any of the files changed unexpectedly? Are any files missing?

Be more cautious as you get nearer the release

Try not to make major changes to your code or toolchain near a release. It is too risky and it means lots of extra testing. Sometimes it is better to ship a release with a minor bug than fix it near the release and risk causing a much worse problem that might not get detected in testing.

Test your release on a clean machine

Most of us have probably sent out a release that didn’t work on a customer’s machine due to a missing dynamic library. Oops. Make sure you test your release on a non-development machine. VMs are useful for this. Don’t expect customers to be very impressed when you tell them ‘It works on my machine‘.

Test on a representative range of platforms

At least run a smoke test on the oldest and most recent version of each operating system you support.

Automate the testing where you can

Use unit tests and test harnesses to automate testing where practical. For example I can build a command line version of the seating optimization engine for my table plan software and run it on hundreds of sample seating plans overnight, to test changes haven’t broken anything.

If you set up a continuous integration server you can build a release and test it daily or even every commit. You can then quickly spot issues as soon as they appear. This makes bug fixing a lot easier than trying to work out what went wrong weeks down the line.

But still do manual testing

Automated test won’t pick up everything, especially with graphical user interface issues. So you still need to do manual testing. I find it is very useful to see real-time path coverage data during testing, for which I use Coverage Validator.

Use third parties

You can’t properly test your own software or proof read your own documentation any more than you can tickle yourself. So try to get other people involved. I have found that it is sometimes useful to pay testing companies to do additional testing. But I always do this in addition to (not instead of) my own testing.

Your documentation is an important part of the release. So make sure you get it proof read by someone different to the person that wrote it.

Use your customers

Even two computers with the same hardware specifications and operating system can be set up with an almost infinite range of user options (e.g. screen resolutions, mouse and language settings) and third party software (e.g. anti-virus). Getting customers involved in beta testing means you can cover a much wider range of setups.

When I am putting out a major new release I invite customers to join a beta mailing list and email them each time there is a new version they can test. In the past I have offered free upgrades to the customers who found the most bugs.

Don’t rely only on testing

I believe in a defence in depth approach to QA. Testing is just one element.

Automate the release process as much as you can

Typically a release process involves quite a few steps: building the executable, copying files, building the installer, adding a digital signature etc. Write a script to automate as much of this as possible. This saves time and reduces the likelihood of errors.

Use a checklist for everything else

There are typically lots of tasks that can’t be automated, such as writing release notes, updating the online FAQ, writing a newsletter etc. Create a comprehensive checklist that covers all these tasks and go through it every release. Whenever you make a mistake, add an item to the checklist to catch it next time. Here is a delightfully meta checklist for checklists.

Write release notes

Customers are entitled to know what changes are in a release before they decide whether to install it. So write some release notes describing the changes. Use screen captures and/or videos, where appropriate, to break up the text. Release notes can also be very useful for yourself later on.

Email customers whose issues you have fixed

Whenever I record a customer bug report or a feature request, I also record the email of the customer. I then email them when there is a release with a fix. It seems only polite when they have taken the effort to contact me. But it also encourages them to report bugs and suggest features in future. I will also let them access the release before I make it public, so they can let me know if there are any problems with the fix that I might not have spotted.

Don’t force people to upgrade

Don’t force customers to upgrade if I don’t want to. And don’t nag them every day if they don’t. A case in point is Skype. It has (predictably) turned from a great piece of software into a piece of crap now that Microsoft have purchased it. Every release is worst than the last. And, to add insult to injury, it just keeps bleating at me to upgrade and there doesn’t seem to be any way to turn off the notifications.

Don’t promise ship dates

If you promise a ship date and you get your estimate wrong (which you will) then either:

You have ship software that isn’t finished; or
You miss your ship date

Neither are good. So don’t promise ship dates. I never do and it makes my life a lot less stressful. It’s ready when it’s ready. I realize that some companies with investors, business partners and large marketing departments don’t have that luxury. I’m just glad that I am not them.

Inform existing customers of the release

There isn’t much point in putting out releases if no-one knows about them. By default my software checks an XML file on my server weekly and informs the customer if a new update is available. I also send out a newsletter with each software release. I generally get a spike in upgrades after each newsletter.

Don’t release too often

Creating a stable release is a lot of work, even if you manage to automate some of it. The more releases you do, the higher percentage of your time you will spend testing, proof reading and updating your website.

Adobe Acrobat seems to go through phases of nagging at me almost daily for updates. Do I think “Wow, I am so happy that those Adobe engineers keep putting out releases of their useful free software”? No. I hate them for it. If you have an early stage product with early-adopters, they may be ok with an update every few days. But most mainstream customers won’t thank you for it.

Don’t release too infrequently

Fixing a bug or usability issue doesn’t help the customer until you ship it. Also a product with very infrequent updates looks dead. The appropriate release frequency will vary with the type of product and how complex and mature it is.

Digitally sign your releases

Digital certificates are a rip-off. But unsigned software makes you look like an an amateur. I am wary of downloading any software that isn’t digitally signed. Apple now prevents you downloading unsigned software by default. Signing is just an extra line in your build script. It is a bit tedious getting a digital certificate though, so get one that lasts several years.

Check your binaries against major anti-virus software

Over zealous anti-virus software can be a real headache for developers of downloadable software. So it is worth checking if your release is likely to get flagged. You can do this using free online resource virustotal.com. If you are flagged, contact the vendor and ask them to whitelist you.

‘The perfect is the enemy of the good’

Beware second system effect. If you wait for perfection, then you will never ship anything. As long as this release is a significant improvement on the last release, then it is good enough to ship.

Pace yourself

Creating a release is exhausting. Even maths, physics and software prodigy Stephen Wolfram of Mathematica says so:

I’ve led a few dozen major software releases in my life. And one might think that by now I’d have got to the point where doing a software release would just be a calm and straightforward process. But it never is. Perhaps it’s because we’re always trying to do majorly new and innovative things. Or perhaps it’s just the nature of such projects. But I’ve found that to get the project done to the quality level I want always requires a remarkable degree of personal intensity. Yes, at least in the case of our company, there are always extremely talented people working on the project. But somehow there are always things to do that nobody expected, and it takes a lot of energy, focus and pushing to get them all together.

So look after yourself. Make sure you get enough sleep, exercise and eat healthily. Also things may be at their most intense straight after the release with promotion, support, bug fixing etc. So it may be a good idea to take a day or two off before you send the release out.

Don’t release anything just before you go away

There is always a chance a new release is going to mess things up. If you are a one-man band like me, you really don’t want to make a software release just before you go away on holiday or to a conference. Wait until you get back!

Fix screwups ASAP

We all make mistakes from time to time. I recently put out a release of my card planning software, Hyper Plan, that crashed on start-up on some older versions of macOS. Oops. But I got out a release with a fix as soon as I could.

Treat yourself after a release

Releases are hard work. A successful release deserves a treat!

Anything I missed?

Things you don’t need for v1.0

Few people launch software products expecting them to fail. But many products do fail. I don’t have any figures, but I think I can fairly confidently state that more commercial software products fail than succeed. You think your product isn’t going to be one of the failures. But so does everyone else. The only way to find out for sure is to launch. The sooner you launch, the sooner you will find out. I have banged the drum for releasing early before, so I won’t labour it here. But it begs the question – how do I launch fast? What do I leave out? Based on my experiences of launching 3 software products, this is what I would leave out.

Polish

As developers we (hopefully) all want to do great work that we can feel proud of. But, as entrepreneurs, we need to be careful not to spend lots of time polishing something that might be a turd. So ship v1.0 before it is polished. Early adopters tend to be fairly forgiving of a few rough edges, if they are interested in the direction you are taking. I spent 6 months (part-time) working on the first version of my AdWords keyword tool. It flopped. So I shipped the first version of my visual planning software within a few weeks of writing the first line of code. It was pretty bare-bones and a bit slow for plans with hundreds of cards, but it was enough to demonstrate the basic concept.

Designer website

You don’t need a beautiful, state-of-the-art website to launch your product. My own table planner software had a pretty ropey website (designed by me) for the first 10 years and it did fine. Just make sure the website clearly conveys what your product does.

Logo

You don’t need a professional logo for v1.0. The product name in coloured text using a font other than Arial will probably be fine. I did the initial logo for Hyper Plan in Microsoft Word Art in 10 minutes. Here it is in all it’s glory:

I only paid a designer to come up with something better once I was sure it was worth my while.

DRM/Payment processing

I shipped the first version of Hyper Plan without even setting up licensing or payment processing. Every time you ran it, it just put up a window saying that it would expire on a certain date and that a new release would be available by that date. After that date it just stopped working.

I only added licensing and payment processing once I had proved enough people were interested in the concept to make it worth my while. If you are going to take this approach, make sure you let people know that they will be expected to pay at some point.

Sophisticated pricing model

Ideally you want to segment your customers so you can charge more for the people who are prepared to pay more. But you probably don’t understand your market well enough to do this when you are starting out. So just pick a single price. I introduced segmented pricing for PerfectTablePlan in v4. Hyper Plan still has a single price.

Feature parity with your competitors

Trying to achieve feature parity with established competitors in v1.0 is a fool’s errand. Just pick one pain point that you think is not being well addressed and try to solve that. Make your lack of features a selling point by emphasizing how simple your product is to use.

Multi Platform

If it is going to take significant additional effort to release multi-platform, then just pick one platform to launch v1.0 on.

Extensive documentation

The first version of your product should be simple enough and well enough designed that it doesn’t need extensive documentation. My Hyper Plan software has been out for a year and it still only has a one page quick start guide.

Mailing list

Many people advocate building up a mailing list of interested people before you launch. It obviously helps a lot if you already have an audience in the market you are launching into. But, if you don’t, it takes significant time and effort to build that audience. I would rather put in that effort once I have something to show them.

Trademark

Why bother to spend time and money trademarking something if you don’t even know if anyone wants it?

Patent

I’m not a fan of software patents and I don’t have any patents after nearly 11 years in business. So I certainly wouldn’t waste time and money on a patent for v1.0.

Lawyers

If a bug in your software could kill someone or destroy their business, you should probably talk to a lawyer. Otherwise a boiler-plate end user licence agreement is probably fine for v1.0.

Company

I did create a limited company before I launched my first product to get a bit of extra legal protection. But its not strictly necessary (in the UK at least).

Trade-offs

It’s all a tradeoff. Obviously it is better to have a beautiful website than an ugly one. But is it worth spending lots of time and money on designing a beautiful website for an unproven product?

The best approach depends very much on your market and circumstances. If you are a big player with lots of money and reputation, then much of the above may not apply. If you are selling web design products, you had better have a pretty slick looking website for v1.0. If you are selling aircraft avionics systems then I hope v1.0 of your product is pretty polished.

New shiny thing

nz20131170 I’m working on a new product. I’ve done all the interesting creative work, such as designing the user interface and implementing the difficult algorithmic parts. Now I’m left with the boring stuff, the installer, website, documentation, licensing etc. The things that make it a proper commercial product. They aren’t optional.

Suddenly I am thinking of all sorts of other interesting ideas for products I would rather be working on. The temptation is strong. The new ideas seem so much more exciting than what I am working on now. But it is a mirage. These shiny new ideas would also require their fair share of drudge work to ship.

Continually abandoning work in progress for a new idea is also a form of cowardice. If I never ship, then I can’t fail. But I can’t succeed either. And I won’t learn anything useful from a string of half-finished products that never shipped. So I just have to push on through the tedious bit, knowing that things will get more interesting again once I ship. In my weaker moments I sit down and sketch out ideas for new products on big sheets of paper. Then I file them away and get back to work.

It isn’t easy to stay focussed week after week and put in the hard work to create a new product for uncertain rewards. It’s what separates the professionals from the amateurs. Professionals ship. Now excuse me, I have a product to finish.