Tag Archives: tools

Visual vs text based programming, which is better?

Visual programming tools (also called ‘no-code’ or ‘low-code’) have been getting a lot of press recently. This, in turn, has generated a lot of discussion about whether visual or text based programming (coding) is ‘best’. As someone who uses text programming (C++) to create a visual programming data wrangling tool (Easy Data Transform) I have some skin in this game and have thought about it quite a bit.

At some level, everything is visual. Text is still visual (glyphs). By visual programming here I specifically mean software that allows you to program using nodes (boxes) and vertexes (arrows), laid out on a virtual canvas using drag and drop.

A famous example of this sort of drag and drop visual programming is Yahoo Pipes:

But there are many others, including my own Easy Data Transform:

Note that I’m not talking about Excel, Scratch or drag and drop GUI designers. Although some of the discussion might apply to them.

By text programming, I mean mainstream programming languages such as Python, Javascript or C++, and associated tools. Here is the QtCreator Interactive Development Environment (IDE) that I use to write C++ in, to create Easy Data Transform:

The advantages of visual programming are:

Intuitive. Humans are very visual creatures. A lot of our brain is given over to visual processing and our visual processing bandwidth is high. Look at pretty much any whiteboard, at any company, and there is a good chance you will see boxes and arrows. Even in non-techie companies.
Quicker to get started. Drag and drop tools can allow you to start solving problems in minutes.
Higher level abstractions. Which means you can work faster (assuming they are the right abstractions).
Less hidden state. The connections between nodes are shown on screen, rather than you having to build an internal model in your own memory.
Less configuration. The system components work together without modification.
No syntax to remember. Which means it is less arcane for people who aren’t experienced programmers.
Less run-time errors, because the system generally won’t let you do anything invalid. You don’t have to worry about getting function names or parameter ordering and types right.
Immediate feedback on every action. No need to compile and run.

The advantages of text programming are:

Denser representation of information.
Greater flexibility. Easier to do things like looping and recursion.
Better tooling. There is a vast ecosystem of tools for manipulating text, such as editors and version control systems.
Less lock-in. You can generally move your C++ or Python code from one IDE to another without much problem.
More opportunities for optimization. Because you have lower-level access there is more scope to optimize speed and/or memory as required.

The advantages and disadvantages of each are two sides of the same coin. A higher level of abstraction makes things simpler, but also reduces the expressiveness and flexibility. The explicit showing of connections can make things clearer, but can also increase on-screen clutter.

The typical complaints you hear online about visual programming systems are:

It makes 95% of things easy and 5% of things impossible

Visual programming systems are not as flexible. However many visual programming systems will let you drop down into text programming, when required, to implement that additional 5%.

Jokes aside, I think this hybrid approach does a lot to combine the strengths of both approaches.

It doesn’t scale to complex systems

Managing complex systems has been much improved over the years in text programming, using techniques such as hierarchy and encapsulation. But there is no reason these same techniques can’t also be applied to visual programming.

It isn’t high enough performance

The creators of a visual programming system are making a lot of design decisions for you. If you need to tune a system for high performance on a particular problem, then you probably need the low level control that text based programming allows. But with most problems you probably don’t care if it takes a few extra seconds to run, if you can do the programming in a fraction of the time. Also, a lot of visual programming systems are pretty fast. Easy Data Transform can join 2 one million row datasets on a laptop in ~5 seconds, which is faster than base R.

It ends up as spaghetti

Unreal Blueprint spaghetti from reddit.com/r/ProgrammerHumor/

I’m sure we’ve all seen examples of spaghetti diagrams. But you can also create horrible spaghetti code with text programming. Also, being able to immediately see that a visual program has been sloppily constructed might serve as a useful cue.

If you are careful to layout your nodes, you can keep things manageable (ravioli, rather than spaghetti). But it starts to become tricky when you have 50+ nodes with a moderate to high degree of connectivity, especially if there is no support for hierarchy (nodes within nodes).

Automatic layout of graphs for easier comprehension (e.g. to minimize line crossings) is hard (NP-complete, in the same class of problems as the ‘travelling salesman’).

No support for versioning

It is possible to version visual programming tools if they store the information in a text based file (e.g XML). Trying to diff raw XML isn’t ideal, but some visual based programming tools do have built-in diff and merge tools.

It isn’t searchable

There is no reason why visual programming tools should not be searchable.

Too much mousing

Professional programmers love their keyboard shortcuts. But there is no reason why visual programming tools can’t also make good use of keyboard shortcuts.

Vendor lock-in

Many visual programming tools are proprietary, which means the cost can be high for switching from one to another. So, if you are going to invest time and/or money heavily in a visual programming tool, take time to make a good choice and consider how you could move away from it if you need to. If you are doing quick and dirty one-offs to solve a particular problem that you don’t need to solve again, then this doesn’t really matter.

‘No code’ just means ‘someone else’s code’

If you are using Python+Pandas or R instead of Easy Data Transform, then you are also balancing on top of an enormous pile of someone else’s code.

We are experts, we don’t need no stinkin drag and drop

If you are an experienced text programmer, then you aren’t really the target market for these tools. Easy Data Transform is aimed at the analyst or business guy trying to wrangle a motley collection of Excel and CSV files, not the professional data scientist who dreams in R or Pandas. However even a professional code jockey might find visual tools faster for some jobs.

Both visual and text programming have their places. Visual programming is excellent for exploratory work and prototyping. Text based programming is almost always a better choice for experts creating production systems where performance is important. When I want to analyse some sales data, I use Easy Data Transform. But when I work on Easy Data Transform itself, I use C++.

Text programming is more mature than visual programming. FORTRAN appeared in the 1950s. Applications with graphical user interfaces only started becoming mainstream in the 1980s. Some of the shortcomings with visual programming reflect it’s relative lack of maturity and I think we can expect to see continued improvements in the tooling associated with visual programming.

Visual programming works best in specific domains, such as:

3d graphics and animations
image processing
audio processing
game design
data wrangling

These domains tend to have:

A single, well defined data type. Such as a table of data (dataframe) for data wrangling.
Well defined abstractions. Such as join, to merge 2 tables of data using a common key column.
A relatively straightforward control flow. Typically a step-by-step pipeline, without loops, recursion or complex control flow.

My teenage son has been able to do some (I think) pretty impressive 3D modelling and animations just with Blender’s visual tools.

Visual programming has been much less successful when applied to generic programming, where you need lots of different data types, a wide range of abstractions and potentially complex control flow.

I’ve been a professional software developer since 1987. People (mostly in marketing) have talked about replacing code and programmers with point and click tools for much of that time. That is clearly not going to happen. Text programming is the best approach for some kinds of problems and will remain so for the foreseeable future. But domain-specific visual programming can be very powerful and has a much lower barrier to entry. Visual programming empowers people to do things that might be out of their reach with text programming and might never get done if they have to wait for the IT department to do it.

So, unsurprisingly, the answer to ‘which is better?’ is very much ‘it depends’. Both have their place and neither is going away.

Choosing a market for your software

The efficient market hypothesis states that “asset prices fully reflect all available information”. If the efficient market hypothesis is true, then you would expect actively managed funds (where fund managers pick the stocks) to do no better than index funds. That does seem to be the case:

“Numerous studies have shown that index funds, with their low costs and ability to closely mimic the returns of markets both broad and narrow, steadily outperform the returns of most actively managed funds.” Wall Street Journal

Unless you have some sort of insider knowledge (which it might be illegal to exploit), you might as well invest in index funds or get your cat to pick your stocks as pay someone else to do it.

But I am interested in a different sort of market efficiency. If you have to pick a vertical market to start a software business in, does it matter which vertical market you pick? If the market is perfectly efficient for businesses, then each vertical will have a level of competition proportional to the size of the market. In that case you should have an equal chance of success whether you decide to write a game, a developer tool, an anti-virus product or a CRM system.

From lots of reading and talking to other software business owners I have come to the conclusion that the market is highly inefficient for businesses. The market vertical you pick has a big effect on your chances of success. It seems to me that the three worst verticals are: games, developer tools and consumer mobile apps.

Games are fun! Writing a game sounds like a blast. Much more exciting than writing software for boring businesses. It has also been getting easier to write games due to the ever improving tools. Consequently, the market for games is totally saturated. The outlook for independent games developers looks grim. Today on the Steam platform there are 12,971 games listed. Even some of the big and famous games developers only seem to survive by forcing their staff to work vast amounts of unpaid overtime.

Pretty much every software entrepreneur has considered creating a software development tool at some point. I know I have. It is a market that we all understand (or think we do). But consequently it is saturated. Software developers are also pretty horrible customers. They are used to using lots of free software. And that tool you spent years developing? They think they can write something better over a weekend.

“Thousands of people used RethinkDB, often in business contexts, but most were willing to pay less for the lifetime of usage than the price of a single Starbucks coffee (which is to say, they weren’t willing to pay anything at all). … Developers love building developer tools, often for free. So while there is massive demand, the supply vastly outstrips it. This drives the number of alternatives up, and the prices down to zero.” Why RethinkDB failed

I wrote back in 2010 what a horrible market the iPhone app store is for developers. Since then the number of apps has increased tenfold to 2.2 million, the average paid app price is a measly $1.01 ($0.48 for games) and some 90%+ of apps are free or freemium.

You should be wary of markets with no competition. But the really high levels of competition in these three markets drives down prices and makes it very hard to get noticed. Obviously not everyone in these 3 markets is failing. It is possible to create a product in one of these markets and be wildly successful (Indie game developer Notch of Minecraft fame springs to mind). But I think the odds are very much stacked against you.

So what market should you pick to maximize your chances of commercial success? Aside from the obvious factors (e.g. something you are interested in and knowledgeable about, something that solves a real problem etc) I suggest avoiding anything considered ‘sexy’ by other developers.

Here is a radical idea – create a software product aimed at women. The vast majority of software is written by men and consequently it tends to cater for men. 50% of the world’s population are women and they buy software too!

Just because a product is not in a ‘sexy’ market doesn’t mean that it has to be boring to create. I have found plenty of interesting usability, optimization and visualization problems to solve while developing my own seating planning and visual planning software products.

Here is a thought experiment. Imagine you are talking to another software guy at a conference and explaining what you product does. If your imaginary software guy says “that sounds cool”, then it’s probably a tough market to create a commercial product in. But if they look a bit surprised or their eyes glaze over, then you might be on to something.

80 useful tools and services for software businesses

Some of the most useful nuggets of information I come across in blogs and podcasts are mentions of tools and services used by other people to better run their software businesses. So I have put together my own list of useful tools and services to run a software business.

Feel free to recommend your own favourites in the comments below. Please include your relationship to the tool/service (e.g. customer, user, employee or owner). You can also comment below about your experiences (positive or negative) with any of the tools and services listed. Anonymous comments will be treated with suspicion and may be deleted

Animated GIFs

The human brain and visual system is highly optimised to detect movement. If you don’t believe me, watch what happens to people’s attention when you turn on a TV in a room. Even if the sound is off, the program is dull and the conversation is interesting, people will find it very hard not to stare at the TV. You can exploit this by using animation on your website to grab the user’s attention. Animation is also a useful way of packing a lot of content into a limited space on your web page.

Animated GIFs are a useful low-tech way of adding animation to a website. They work in pretty much any browser, without requiring visitors to download a plug-in or even click a ‘play’ button. I use them on the PerfectTablePlan home page to show rotating testimonials and on adwords landing pages to give a brief visual overview of what PerfectTablePlan can do.

Animated GIFs are quite easy to create. Here is how I created the image above (on Windows):

I used Sizer (freeware) to size the PerfectTablePlan main window to 960×750.
I used SnagIt (commercial) to capture various screenshots, resize them to 320×250 and save them as separate 7-bit GIFs.
I dragged the GIFs onto UnFreez (donationware) and created an animated GIF. (You can also use Adobe Photoshop, if you have it).
I dragged the animated GIF onto SuperGIF (commercial with trial) to reduce the file size (by about 5% in this case).

The final result isn’t a work of art, but it is hopefully enough to grab the visitors attention and whet their appetite for more information.

Animated GIFs can get very large if you aren’t careful. But it rather defeats the object if your website visitor clicks ‘back’ before the image has loaded. I used 7-bit GIFs, small image dimensions, a limited number of frames and GIF optimisation to keep the file above to 72kb.

A word of warning – use animation sparingly or the effect can be quite overwhelming (don’t click this link if have epilepsy or a refined sense of taste).