Tag Archives: software

Experiences promoting niche software

Leave a reply

This is a guest post from fellow software developer, Simon Kravis.

It’s sometimes said that software development is only 10% of what’s required to earn money from software and I can attest to that. Since 2018 I have been developing photo captioning and related software, more as a retirement diversion than a serious source of income (after a career mostly involved in writing scientific and engineering analysis software), in the hope that sales income would at least cover running costs. My best marketing tool has been writing reviews of the class of software that I produce, and the hosting site (Hub Pages) provides some useful analytics on how often these are accessed and for how long. Below is the graph for an article on tagging.

The decline since early 2022 is hard to explain – the article is periodically updated so the steady decline is not due to diminishing ‘freshness’ – which for Google is probably a file Modified date.

Here is another review article profile (Scanning Multiple Photos) showing a similar decline:

But another (Best Photo Captioning Software) has held up, though at a low level.

I offer digital photo captioning software (Caption Pro) on Windows and Mac platforms, and an iPhone captioning app (CaptionEdit), with the Windows version dating back to 2017. I also offer part of the functionality of Caption Pro on Windows for auto-cropping scans of multiple paper photos (ImageSplit). On Windows neither Caption Pro software downloads or sales seem to correlate with review accesses, despite about 1/3 of web site accesses coming from the review. However, downloads do show some correlation with Caption Pro web site sessions, as shown in the graph below.

Sales do not correlate with downloads, which perhaps explains why most advertising for niche products is not successful – it may increase downloads but this does not appear to increase sales. The observed proportion of downloads resulting in sales for ImageSplit and Caption Pro are 6% and 9% respectively. The lack of correlation between sales and downloads may be due to the small number of sales per month, which results in random fluctuation dominating the results.

The decision to enter the Apple “Walled Garden” of software was partly at the prompting of friends rather than a commercial evaluation. Apple Developer membership (costing ~US$100 per year) is required to prevent software being blocked from installation through being from an unknown publisher. Further costs were purchasing a fairly modern Mac on which to perform development, as the App Store will only accept software developed using recent versions of the Xcode development environment, which will only run on fairly recent hardware. The App Store takes a commission of 15% on sales, which is quite reasonable when compared to the difficulty of implementing e-commerce on Windows, where a PayPal account eases the problem of low-value foreign-currency transactions, but e-commerce plug-ins may stop working after years for no discernible reason. The review process for software acceptance into the App Store is generally fast, but seemly trivial issues can require resubmission. Features which have passed one review may be rejected in a later one. The review process is generally fast, but on one occasion took 4 weeks.

Caption Pro for Mac has been available (via the App Store) only since Sep 2021.It appears within the top 6 results for a search using “Caption Photos”, which is the source for most downloads. About 3.5% of downloads result in sales. This figure is much less than the Windows version of the same app, despite Mac users’ reputation for being more willing to pay for software. The iPhone app did not appear at all initially when searching for “Caption Photos” in the App Store. After 6 months it began appearing as result number 140, after it had 360 downloads. This poor ranking performance is probably because “Caption Photos” is a very popular keyword used by many apps, including those that only caption videos. It has had very few downloads and sales, despite Apple Search Ads and Apptimizer campaigns. The number of downloads increased dramatically during the Apptimizer campaign between Jan 24 and Feb 2 (as they were purchased) but the change in ranking from these downloads did not result in any sales, perhaps because no installs were purchased. The Apple search ads campaign (which resulted in the app being shown as an ad 1 in 50 times when the search phrase “Caption Photos” was used) did not greatly affect downloads or sales. A Facebook ad campaign to show a link to the app whenever “Genealogy” or “Genealogy Software” was searched for was also unsuccessful, and very expensive, as Facebook charges by impressions rather than clicks. Additional backlinks to the web site were purchased in September 2022 from Links Management in an attempt to improve the web site Google ranking, but this did not appear to have any effect on web traffic.

Mac and Windows users contacting me with problems have had a wide range of experience level – from completely naïve to former programmers. Most have been from the US, which reflects the geographic distribution of sales. There have many downloads to non-English speaking countries but very few sales.

Some results from the Mac and iPhone Apps are shown below:

On balance, developing for Apple platforms was not a good commercial decision, as the advantages of a mostly captive audience (completely captive in the case of the iPhone) do not seem to result in higher rates of downloads or sales. Competition for iPhone apps is so intense that niche products without massive advertising budgets are unlikely to succeed. The same is likely to apply to Android phone apps, which anecdotally have a less rigorous review process. My experience is that advertising and backlink purchase for any platform are not effective in increasing sales for niche software.

Simon Kravis runs Aleka Consulting, a small software and consultancy company in Canberra, Australia specializing in information management and offering a number of software products. He has mainly developed scientific and engineering programs, starting in the era of paper tape.

Why you can’t parse CSV with a regular expression

14 Replies

Regular expressions are a very useful tool in a programmer’s toolbox. But they can’t do everything. And one of the things they can’t do is to reliably parse CSV (comma separated value) files. This is because a regular expression doesn’t store state. You need a state machine (or something equivalent) to parse a CSV file.

For example, consider this (very short) CSV file (3 double quotes + 1 comma + 3 double quotes):

“””,”””

This is correctly interpreted as:

quote to start the data value + escaped quote + comma + escaped quote + quote to end the data value

E.g. a single value of:

“,”

How each character is interpreteted depends on what characters come before and after it. E.g. the first quote puts you into an ‘inside data’ state. The second quote puts you into a ‘might be an escaped for the following character or might be end of data’ state. The third quote puts you back into a ‘inside data’ state.

No matter how complicated a regex you come up with, it will always be possible to create a CSV file that your regex can’t correctly parse. And once the parsing goes wrong, everything after that point is probably garbage.

You can write a regex that can handle CSV file where you are guaranteed there are no commas, quotes or carriage returns in the data values. But commas, quotes or carriage returns in the data values are perfectly valid in CSV files. So it is only ever going to handle a subset of all the possible well-formed CSV files.

Note that you can parse a TSV (tab separated value) file with a regex, as TSV files are (generally!) not allowed to contain tabs or carriage returns in data and therefore don’t need escaping.

See also on Stackoverflow:

Using regular expressions to parse HTML: why not?

Adventures in content marketing

4 Replies

Back in 2011 I created eventcountdown.com. It had a snazzy downloadable, PerfectTablePlan-branded countdown clock for Windows and web-based countdown clock with ads for PerfectTablePlan. Both free. The idea was people searching for countdown clocks for events (such as their wedding) would find the site via Google, find out about PerfectTablePlan and a certain percentage would then buy my event seating planner software.

I paid other people to create the Windows and web versions of the countdown clock. The web-based clock was updated from time to time to add pre-built countdowns for events like superbowl, the olympics, christmas, thanksgiving etc. And I fielded the occasional support emails related to the Windows countdown clock.

This is the total traffic to the site from 2011 to 2023:

The peaks are mostly due to superbowl. The site got 38k hits in a single day just before superbowl 2019! The free Windows countdown clock also drew quite a lot of traffic. In total the site got some 1.7 million page views over 12 years. Only a small percentage of these visitors clicked through to PerfectTablePlan.com, but still a useful number. Perhaps some people were also prompted to investigate PerfectTablePlan by the branding on the downloadable clock. The site might have also had some SEO benefits for PerfectTablePlan.com. Who knows.

The eventcountdown.com website is now gone (the domain redirects to PerfectTablePlan.com). It didn’t seem worth the effort to keep adding events to the web countdown clock with the traffic now so low. Also both the website and windows clock were looking dated. But I think it was a worthwhile investment of my time and money.

I have also created various other contents pages and mini-sites over the years: articles on table planning, font collections, free clipart, place card templates etc. You can see similar trajectories for some of those.

The traffic seems to reach a peak after 3-7 years and then slowly decay away. Although I have shown them with the same vertical scale here, some generated a lot more traffic than others.

I did some basic on-page SEO for these content pages. For example, looking at Adwords keyword data to choose the page title and H1. But nothing beyond that. No paid promotion or backlink building campaigns.

I tried paying people to write articles related to events. But none of these ever generated any worthwhile traffic. Google could somehow smell the insincerity.

For my data cleaning software product I have been concentrating on ‘how to’ pages and supporting videos aimed at specific topics. These are intended to both help existing customers and to attract new traffic. For example, how to clean data. I have also been posting these videos on the Easy Data Transform YouTube channel. The numbers of hits monthly on the Youtube videos are relatively low, but they are quite targeted and hopefully will be generating traffic for years to come.

So content marketing take-aways based on my experience are:

  • Free content can be a useful way to bring free traffic to your website.
  • The amount of traffic you get is quite hit and miss. Some content has generated a lot more traffic than expected, some a lot less.
  • The content needs to be well targeted if you want to have any chance of converting it to sales.
  • Google will grow bored of it eventually. You might be able to increase the longevity by updating the content. I’ve not been very diligent with this, but even neglected content pages can generate useful traffic over 10+ year lifespan.

Renewing my authenticode digital certificate

22 Replies

The authenticode digital certificate I bought back in 2019 expired recently, so I had to get a new certificate (you can’t renew a certificate, as such, you just need to buy a new one). A few months before the expiry I emailed KSoftware.net, who I had bought previous digital certificates from and with whom I had always had a good experience in the past. No reply. I tried a couple more times, including the personal email of Mitchell, the founder. Nothing. Someone else told me they had had similar experiences. Their recent trustpilot ratings are a horror show. And the copyright date on their website is ‘2003 – 2021’. But they were still advertising on Google Adwords. I have no idea what is happening here. If you are reading this Mitchell, I hope you are ok.

With KSoftware out of the picture I looked elsewhere. Eventually I ended up buying a new Sectigo certificate from signmycode.com. I partly chose them because they offered a 5 year certficate and the less often I have to go through the ball ache of a getting a new certificate, the better. The experience was decidely mixed.

The good:

  • The prices seem reasonable, compared to other options.
  • Support was responsive. English didn’t appear to be their first language, but it was good enough.
  • I got my new certificate within a few days and have had no issues with it so far. The change in certificate seems to be set off a few customer’s anti-virus software, but that was to be expected.

The mediocre:

  • The online guidance and documentation on the process was mediocre, at best.
  • I was a bit confused about whether I had to click ‘Buy now’ or ‘Renew now’. It seems this is more marketing/SEO purposes and it doesn’t matter which you click.
  • I had to send them a photo of me holding a government ID. This felt pretty uncomfortable, but might be something mandated by the certificate companies.

The bad:

  • After I got my certificate I checked the expiry date and it was only 3 years. When I queried this I was rold that the ‘5 year certificate’ I thought I had bought is not a 5 year certificate. It is a 3 year certificate, then I have to apply for a new pre-paid 2 year certificate in 3 years time.

This is what you see when you click on ‘Buy now’:

When you see this, wouldn’t you expect to get a single 5 year certificate? If there was anything explaining that this was 2 separate certificates, I didn’t notice it. It certainly didn’t mention it on their home page. This feels deceptive to me.

Who knows if this company will still be there in 3 years time? I emailed them and told them I wanted to keep the new 3 year certificate and for them to refund the 2 year certificate. They said they would only refund the entire order and then I would have to start the whole process all over again. They also claimed:

“renewal validation is much more easy then buying a new certificate as most of the validation part is getting carry forward.”

We’ll see. Buyer beware.

See also: The great digital certificate ripoff?

** Update 08-Mar-2023 **

Michell of KSoftware has contact me to say that he is alive and kicking. Read the comments below for more details.

Winterfest 2022

Easy Data Transform and Hyper Plan Professional edition are both on sale for 25% off at Winterfest 2022. So now might be a good time to give them a try (both have free trials). There is also some other great products from other small vendors on sale, including Tinderbox, Scrivener and Devonthink. Some of the software is Mac only, but Easy Data Transform and Hyper Plan are available for both Mac and Windows (one license covers both OSs).

12 rules for software business happiness

Here are a few rules for happiness that I have learned (often the hard way) running a solo software business since 2005

Make sure your important stuff is backed-up automatically

Any sort of manual back-up is going to get forgotten. Back-up to more than one place, at least one of which is offsite.

Stay away from the bleeding edge

Stick with tried and trusty tools and technologies, where you can. JQuery will probably be here in another 10 years, but the latest and greatest Javascript framework might not.

Use good suppliers

You need your hosting company, payment processor and other critical suppliers to be rock solid. Think twice about going with a supplier just because they are cheap. Changing suppliers can be a pain, so ask around before trying a supplier.

Use version control for everything important

It matters less which version control system it is. Periodically making a copy of your source folder is not a version control system!

Don’t promise ship dates

Developers are notoriously bad at predicting dates. If you promise a date and get it wrong (and you will) then you either have to miss the date or cut corners. Neither is good.

Never send an email you might later regret

If you are starting to feel angry writing an email, then stop writing. Come back to it later. Or maybe write it, feel a bit better, then delete it without sending.

Write documentation as you go

Few people enjoy writing documentation. But if you leave all the documentation until you have finished programming, then you are likely to rush it and forget stuff.

Have a checklist

Automate where you can. Have checklists for everything else. Keep updating your checklists.

Get someone else to proof read everything

Typos are embarrassing, but it is impossible to proof read your own stuff. So get someone else to proof read any stuff that customers see: web pages, newsletters, documentation etc.

Never release changes just before going on holiday

You don’t want to have to be fire-fighting a new bug when you should be on the beach with your family/friends.

Don’t try to do everything yourself

You could spend weeks learning about taxes, web hosting, CSS or any number of other topics that aren’t central to your business. But why bother? Pay someone who already know this stuff.

Embrace imperfection

If you wait for perfection, then you are never going to ship anything. Just make sure each release is better than the last. Good enough is good enough.

Summerfest 2022

Easy Data Transform and Hyper Plan Professional edition are both on sale for 25% off at Summerfest 2022. So now might be a good time to give them a try (both have free trials). There is also some other great products from other small vendors on sale, including Tinderbox, Scrivener and Devonthink. Some of the software is Mac only, but Easy Data Transform and Hyper Plan are available for both Mac and Windows (one license covers both). Sale ends 12th July.

Why isn’t there a decent file format for tabular data?

Tabular data is everywhere. I support reading and writing tabular data in various formats in all 3 of my software application. It is an important part of my data transformation software. But all the tabular data formats suck. There doesn’t seem to be anything that is reasonably space efficient, simple and quick to parse and text based (not binary) so you can view and edit it with a standard editor.

Most tabular data currently gets exchanged as: CSV, Tab separated, XML, JSON or Excel. And they are all highly sub-optimal for the job.

CSV is a mess. One quote in the wrong place and the file is invalid. It is difficult to parse efficiently using multiple cores, due to the quoting (you can’t start parsing from part way through a file). Different quoting schemes are in use. You don’t know what encoding it is in. Use of separators and line endings are inconsistent (sometimes comma, sometimes semicolon). Writing a parser to handle all the different dialects is not at all trivial. Microsoft Excel and Apple Numbers don’t even agree on how to interpret some edge cases for CSV.

Tab separated is a bit better than CSV. But can’t store tabs and still has issues with line endings, encodings etc.

XML and JSON are tree structures and not suitable for efficiently storing tabular data (plus other issues).

There is Parquet. It is very efficient with it’s columnar storage and compression. But it is binary, so can’t be viewed or edited with standard tools, which is a pain.

Don’t even get me started on Excel’s proprietary, ghastly binary format.

Why can’t we have a format where:

  • Encoding is always UTF-8
  • Values stored in row major order (row 1, row2 etc)
  • Columns are separated by \u001F (ASCII unit separator)
  • Rows are separated by \u001E (ASCII record separator)
  • Er, that’s the entire specification.

No escaping. If you want to put \u001F or \u001E in your data – tough you can’t. Use a different format.

It would be reasonably compact, efficient to parse and easy to manually edit (Notepad++ shows the unit separator as a ‘US’ symbol). You could write a fast parser for it in minutes. Typing \u001F or \u001E in some editors might be a faff, but it is hardly a showstopper.

It could be called something like “unicode separated value” (hat tip to @fakeunicode on Twitter for the name) or “unit separated value” with file extension .usv. Maybe a different extension could used when values are stored in column major order (column1, column 2 etc).

Is there nothing like this already? Maybe there is and I just haven’t heard of it. If not, shouldn’t there be?

And yes I am aware of the relevant XKCD cartoon ( https://xkcd.com/927/ ).

** Edit 4-May-2022 **

“Javascript” -> “JSON” in para 5.

It has been pointed at the above will give you a single line of text in an editor, which is not great for human readability. A quick fix for this would be to make the record delimiter a \u001E character followed by an LF character. Any LF that comes immediately after an \u001E would be ignored when parsing. Any LF not immediately after an \u001E is part of the data. I don’t know about other editors, but it is easy to view and edit in Notepad++.

Positioning Software in a Crowded Market

This is a guest post from serial software entrepreneur Dennis Gurock.

Thinking about product positioning (and matching branding) is especially important if you build a product for a crowded market with many established competitors (and there are many reasons why this can be a good idea). We were in exactly this situation when we initially thought about building and marketing our new test management tool.

Positioning will allow you to better focus on a specific market segment to target, it makes it easier to build a clearer and stronger message to reach customers, and it helps develop the initial product vision and feature set.

What does successful positioning mean for software products? It can mean identifying a unique angle to focus on so you can stand out with your product among other products and competitors. Especially if you are entering a crowded market, this allows you to better communicate the key benefits and features you have to offer. It will help you reach the right customers and ensures that customers remember you when they look for a new product to try.

To come up with positioning for your new product, you can focus on a specific customer segment or niche that you think will be easier to market to or that you think is underserved by existing offerings. It can also help you limit the initial product scope, so you can go to market faster. Then rigorously optimizing for this initial customer segment allows you to establish a market presence and expand to other segments more easily later.

Why is positioning useful?

There are many benefits of coming up with and deciding on positioning for your new software product early on. Once you decide on the positioning, many marketing, product management and sales decisions become more straightforward.

  • Clear message & benefits: it is not easy to stand out in a crowded market. Positioning allows you to come up with clear messaging so you can explain and highlight unique selling points in few words.
  • Target and identify niche/marketing opportunities: it can be difficult to decide which marketing options to try, which campaigns to book and which niches to target. Focusing on a specific market segment based on the product positioning can be a great way to identify matching niches and opportunities.
  • Identify customer fit during sales: one of the most important aspects of the sales process is identifying and ensuring prospects are actually a great fit for your product. It’s wasted time for both you and for your prospects to invest a lot of effort evaluating and piloting a product if they will not benefit from it. Positioning can help you quickly filter and identify which customers to focus on.
  • Better focus on initial product vision: there are a lot of directions to choose when building a new product. If you don’t have a clear vision to guide you, it is easy to be distracted by different directions and work on too many things at the same time. Clear positioning makes it easy to focus product management on specific goals and use cases.
  • Easier to choose features: when you start working with customers, you will (hopefully) receive a lot of feedback on features you should add. Positioning helps you decide which of these features you should actually implement. Often times the most successful products are developed by following strong opinions and saying ‘No’ to many requests.

Examples of software product positioning

Let’s look at a few examples of companies that use positioning to market and build their products. All these examples are from industries and product categories with many existing competitors and products.

  • Testmo: we entered a crowded market with many established testing tools when we developed our new product. Most existing offerings either focus on manual testing, or they offer a complete ALM toolset to handle the entire development lifecycle. With Testmo we had other ideas and wanted to position it differently, focusing on unified testing. This means we combine test cases, automation and exploratory testing in a single platform. At the same time it allows us to limit the scope of the product. We won’t add our own issue tracking, or CI pipelines, or existing DevOps features. Instead of we focus on integrating with other tools customers already use.
  • Another example is the documentation and wiki product GitBook. They heavily focus on software developers and position themselves as the primary tool for developers to publish user docs and to document internal knowledge. With this positioning in mind, they can focus on features that primarily make sense for developers, such as Git synchronization, Markdown support and code snippets. It also allows them to more easily market directly to software developers with a clear message.
  • Then there’s the application monitoring service Checkly. There are many services and products that enable you to monitor apps and sites for downtime and notify you about issues. Checkly positions itself as a tool that enables end-to-end monitoring with flexible scripting. So it doesn’t just make simple web requests to see if a site is still live. It allows customers to write custom scripts to implement complex user flows and thereby not just check if a site is reachable, but also test the entire stack with the front-end, database, authentication and much more. This focus allows them to build more targeted features for advanced use cases and thereby provides more value to customers compared to simpler competitors.
  • The popular email marketing service Campaign Monitor also started with very focused positioning. In the first few years they concentrated on providing the best possible campaign tool for web designers and design agencies. This focus allowed them to invest more in features designers needed, such as white labeling, reusable themes and live email previews. Once they established their market presence, they started to expand their customer base to capture a larger part of the overall market for newsletter tools.

These are just some examples of companies and products that have benefited from clear positioning. Of course there are also countless of examples of companies choosing not to have such clear positioning. There is nothing wrong with this and you can certainly be very successful even if you ignore these points. But more often than not positioning is a useful tool to improve focus on specific goals and customer needs, which increases your chance to build a successful software business.

Dennis Gurock is one of the founders of Testmo, a QA testing tool that unifies test case management with exploratory testing and test automation in one platform. He has been working on products that help teams improve software quality for more than 15 years.