Support is wild in an age of AI

Recently a customer emailed me:

Please consider re adding the option to convert from one time zone to another as was available in [Easy Data Transform] version 1.x. See the attached screen dump.

Easy Data Transform has never had a time zone conversion feature, so that is a bit strange. Although . But the screenshot really set off alarms bells as, despite saying “Easy Data Transform v1.11.2” in the title bar, that is not our software!

I thought that someone was trying to pass of their product as ours and did a search. But I couldn’t find any reference to another piece of software called “Easy Data Transform”. I emailed the customer to ask where they had got the software from. The reply came back:

The screenshot was from a ChatGPT output – maybe hallucinating.

Wow. Not only had ChatGPT hallucinated the feature, but also a fever dream screenshot of the user interface, with the non-existent feature. It looks like a real screenshot at first glance, but the icons are a giveaway if you look closer.

For reference, this is what the actual user interface looks like:

ChatGPT has got some of the input types, transforms and menus correct. But otherwise it looks quite different.

I had another brush with AI hallucinations when I asked MS Copilot how to perform ‘one hot encoding’ in Easy Data Transform. It came back with a very plausible and confident sounding answer, including this summary of the transforms required:

Just one problem – only 1 of these 4 transforms actually exists (Split Col). It hallucinated the other 3!

Customers are increasingly typing questions into AIs, rather than reading documentation or asking on a forum. That is good news if it means that the customer gets a quick and accurate answer without troubling busy developers. But it is very bad news if they are getting incorrect answers, especially when these answers look plausible and are confidently presented. It is galling enough that AIs are stealing all our web traffic, without them giving our customers bad support advice as well! Wild times are ahead.

Ps/ Time zone conversion and one hot encoding are now available in the latest version of Easy Data Transform.

4 thoughts on “Support is wild in an age of AI

  1. Jonathan Matthews's avatarJonathan Matthews

    [With AI guy hat on] I’m kinda surprised that’s happening since you have quite a few How do I? articles. But, that might help get your info. into the model’s weights but these days AI bots are using search grounding more and more.

    Might help with search grounding if you publish pages like “Complete list of transforms available in EDT vx.x”.

    Reply
  2. simonkravis@bigpond.com's avatarsimonkravis@bigpond.com

    Hi Andy

    I’ve been using Copilot to assist in in packaging my software as an msix file which can be placed in the Microsoft Store. The process of converting a 9-year old app packaged using msi to use msix has been very slow, as all kinds of operations I used are impossible for an msix-packaged app. MSIX packaging creates a separate environment for every app, as the iPhone does. Copilot was sort of helpful, but kept telling me how close I was to making a working installer, when in fact implementing its suggestions merely replaced one error with another. After doing everything it suggested, it finally said I should replace Visual Studio 2022 with Visual Studio 2019, which would probably have unleashed another set of errors. Eventually I succeeded but after days of frustrating experimentation. Copilot never hallucinated a screenshot though…

    Now the app (Caption Pro) is in the MS Store I discover that while installs are numerous, use of the software is much rarer – still haven’t has a single one after 20-odd installs, despite about 25% of page viewers installing. This seems to be a known attribute apps in App Stores. Interestingly, the free, limited version (Caption Pro Lite) has had very heavy use

    I had a very similar experience to yours with Reddit advertising … waste of money.

    I do enjoy your posts – keep them up!

    Regards

    Simon Kravis

    Reply
    1. Andy Brice's avatarAndy Brice Post author

      I am currently attempting to get Easy Data Transform into the Windows store.

      They now accept .exe installers, so I didn’t make any changs to Easy Data Transform. Also I am not using their ecommerce.

      My submission is currently awaiting review.

      I am not expecting to get a lot of sales, but I figure it might be a good signal for SEO/AIs.

      Reply

What do you think?