Category Archives: Qt

Creating a Mac Universal binary for Intel and ARM M1/M2 with Qt

Apple has transitioned Macs from Intel to ARM (M1/M2) chips. In the process it has provided an emulation layer (Rosetta2) to ensure that the new ARM Macs can still run applications created for Intel Macs. The emulation works very well, but is quoted to be some 20% slower than running native ARM binaries. That may not seem like a lot, but it is significant on processor intensive applications such as my own data wrangling software, which often processes datasets with millions of rows through complex sequences of merging, splitting, reformatting, filtering and reshaping. Also people who have just spent a small fortune on a shiny new ARM Mac can get grumpy about not having a native ARM binary to run on it. So I have been investigating moving Easy Data Transform from an Intel binary to a Universal (‘fat'[1]) binary containing both Intel and ARM binaries. This is a process familiar from moving my seating planner software for Mac from PowerPC to Intel chips some years ago. Hopefully I will have retired before the next chip change on the Mac.

My software is built on-top of the excellent Qt cross-platfom framework. Qt announced support for Mac Universal binaries in Qt 6.2 and Qt 5.15.9. I am sticking with Qt 5 for now, because it better supports multiple text encodings and because I don’t see any particular advantage to switching to Qt 6 yet. But, there is a wrinkle. Qt 5.15.3 and later are only available to Qt customers with commercial licenses. I want to use the QtCharts component in Easy Data Transform v2, and QtCharts requires a commercial license (or GPL, which is a no-go for me). I also want access to all the latest bug fixes for Qt 5. So I decided to switch from the free LGPL license and buy a commercial Qt license. Thankfully I was eligible for the Qt small business license which is currently $499 per year. The push towards commercial licensing is controversial with Qt developers, but I really appreciate Qt and all the work that goes into it, so I am happy to support the business (not enough to pay the eye-watering fee for a full enterprise license though!).

Moving from producing an Intel binary using LGPL Qt to producing a Universal binary using commercial Qt involved several major stumbling points that took me hours and a lot of googling to sort out. I’m going to spell them out here to save you that pain. You’re welcome.

  • The latest Qt 5 LTS releases are not available via the Qt maintenance tool if you have open source Qt installed. After you buy your commercial licence you need to delete your open source installation and all the associated license files. Here is the information I got from Qt support:
I assume that you were previously using open source version, is that correct?

Qt 5.15.10 should be available through the maintenance tool but it is required to remove the old open source installation completely and also remove the open source license files from your system.

So, first step is to remove the old Qt installation completely. Then remove the old open source licenses which might exist. Instructions for removing the license files:

****************************
Unified installer/maintenancetool/qtcreator will save all licenses (downloaded from the used Qt Account) inside the new qtlicenses.ini file. You need to remove the following files to fully reset the license information.

Windows
"C:/Users/%USERNAME%/AppData/Roaming/Qt/qtlicenses.ini"
"C:/Users/%USERNAME%/AppData/Roaming/Qt/qtaccount.ini"

Linux
"/home/$USERNAME/.local/share/Qt/qtlicenses.ini"
"/home/$USERNAME/.local/share/Qt/qtaccount.ini"

OS X
"/Users/$USERNAME/Library/Application Support/Qt/qtlicenses.ini"
"/Users/$USERNAME/Library/Application Support/Qt/qtaccount.ini"

As a side note: If the files above cannot be found $HOME/.qt-license(Linux/macOS) or %USERPROFILE%\.qt-license(Windows) file is used as a fallback. .qt-license file can be downloaded from Qt Account. https://account.qt.io/licenses
Be sure to name the Qt license file as ".qt-license" and not for example ".qt-license.txt".

***********************************************************************

After removing the old installation and the license files, please download the new online installer via your commercial Qt Account.
You can login there at:
https://login.qt.io/login

After installing Qt with commercial license, it should be able to find the Qt 5.15.10 also through the maintenance tool in addition to online installer.
  • Then you need to download the commercial installer from your online Qt account and reinstall all the Qt versions you need. Gigabytes of it. Time to drink some coffee. A lot of coffee.
  • In your .pro file you need to add:
macx {
QMAKE_APPLE_DEVICE_ARCHS = x86_64 arm64
}
  • Note that the above doubles the build time of your application, so you probably don’t want it set for day to day development.
  • You can use macdeployqt to create your deployable Universal .app but, and this is the critical step that took me hours to work out, you need to use <QtDir>/macos/bin/macdeployqt not <QtDir>/clang_64/bin/macdeployqt . Doh!
  • You can check the .app is Universal using the lipo command, e.g.:
lipo -detailed_info EasyDataTransform.app/Contents/MacOS/EasyDataTransform
  • I was able to use my existing practise of copying extra files (third party libraries, help etc) into the .app file and then digitally signing everything using codesign –deep [2]. Thankfully the only third party library I use apart from Qt (the excellent libXL library for Excel) is available as a Universal framework.
  • I notarize the application, as before.

I did all the above on an Intel iMac using the latest Qt 5 LTS release (Qt 5.15.10) and XCode 13.4 on macOS 12. I then tested it on an ARM MacBook Air. No doubt you can also build Universal binaries on an ARM Mac.

Unsurprisingly the Universal app is substantially larger than the Intel-only version. My Easy Data Transform .dmg file (which also includes a lot of help documentation) went from ~56 MB to ~69 MB. However that is still positively anorexic compared to many bloated modern apps (looking at you Electron).

A couple of tests I did on an ARM MacBook Air showed ~16% improvement in performance. For example joining two 500,000 row x 10 column tables went from 4.5 seconds to 3.8 seconds. Obviously the performance improvement depends on the task and the system. One customer reported batch processing 3,541 JSON Files and writing the results to CSV went from 12.8 to 8.1 seconds, a 37% improvement.

[1] I’m not judging.

[2] Apparently the use of –deep is frowned on by Apple. But it works (for now anyway). Bite me, Apple.

How to add a dark theme to your Qt application

Dark themes are now available for Windows 10 and Mac and it is increasingly expected that desktop applications will offer a dark theme. Previously Qt support for dark themes was patchy. But I am happy to say that it now seems to work fine with Qt 5.12.2, and I have added dark themes to both Windows and Mac versions of my Easy Data Transform and Hyper Plan applications.

Easy Data Transform for Mac with a dark theme:

Easy Data Transform for Windows with a dark theme:

Hyper Plan for Mac with a dark theme:

Hyper Plan for Windows with a dark theme:

I haven’t decided yet whether to add a dark theme to PerfectTablePlan.

Adding dark themes was a fair amount of work. But a lot of that was scouring forums to work out how to integrate with macOS and Windows. Hopefully this article will mean you don’t have to duplicate that work.

Dark themes work a bit differently on Windows and Mac. On Windows changing the UI theme to dark won’t directly affect your Qt application. But you can use an application stylesheet to set the appearance. On Mac changing the UI theme to dark will automatically change your application palette, unless you explicitly block this in your Info.plist file (see below). On both platforms you will need to change any icons you have set to the appropriate light/dark version when the theme changes. Some of this may change in future as dark themes are more closely integrated into Qt on Windows and Mac.

macOS

You can add the following helper functions to a .mm (Objective-C) file:

#include "Mac.h"
#import <Cocoa/Cocoa.h>

bool macDarkThemeAvailable()
{
    if (__builtin_available(macOS 10.14, *))
    {
        return true;
    }
    else
    {
        return false;
    }
}

bool macIsInDarkTheme()
{
    if (__builtin_available(macOS 10.14, *))
    {
        auto appearance = [NSApp.effectiveAppearance bestMatchFromAppearancesWithNames:
                @[ NSAppearanceNameAqua, NSAppearanceNameDarkAqua ]];
        return [appearance isEqualToString:NSAppearanceNameDarkAqua];
    }
    return false;
}

void macSetToDarkTheme()
{
   // https://stackoverflow.com/questions/55925862/how-can-i-set-my-os-x-application-theme-in-code
   if (__builtin_available(macOS 10.14, *))
   {
        [NSApp setAppearance:[NSAppearance appearanceNamed:NSAppearanceNameDarkAqua]];
   }
}

void macSetToLightTheme()
{
    // https://stackoverflow.com/questions/55925862/how-can-i-set-my-os-x-application-theme-in-code
    if (__builtin_available(macOS 10.14, *))
    {
        [NSApp setAppearance:[NSAppearance appearanceNamed:NSAppearanceNameAqua]];
    }
}

void macSetToAutoTheme()
{
    if (__builtin_available(macOS 10.14, *))
    {
        [NSApp setAppearance:nil];
    }
}

The macSetToLightTheme() and macSetToDarkTheme() are useful if you want to give the user the option to ignore the OS theme. Call macSetToAutoTheme() to set it back to the default.

Corresponding header file:

#ifndef MAC_H
#define MAC_H

bool macDarkThemeAvailable();
bool macIsInDarkTheme();
void macSetToDarkTheme();
void macSetToLightTheme();
void macSetToAutoTheme();

#endif // MAC_H

You then need to add these files into your .pro file:

macx {
   ...
   HEADERS += Mac.h
   OBJECTIVE_SOURCES += Mac.mm
}

You can detect a change of theme by overriding changeEvent():

void MainWindow::changeEvent( QEvent* e )
{
#ifdef Q_OS_MACX
    if ( e->type() == QEvent::PaletteChange )
    {
        // update icons to appropriate theme
        ...
    }
#endif
    QMainWindow::changeEvent( e );
}

If you decide you *don’t* want to add a dark theme to your Mac app, the you should add the bold entry below to your Info.plist file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    ...
    <key>NSRequiresAquaSystemAppearance</key>
    <true/>
</dict>
</plist>

This will then force it to be shown in a light theme, regardless of the theme the operating system is in.

Windows

To set a dark theme palette you can use a stylesheet:

QFile f( ":qdarkstyle/style.qss" );
if ( !f.exists() )
{
   qWarning() << "Unable to set dark stylesheet, file not found";
}
else
{
   f.open( QFile::ReadOnly | QFile::Text );
   QTextStream ts( &f );
   getApp()->setStyleSheet( ts.readAll() );
}

The stylesheet I used was a modified version of qdarkstyle from a few years ago.

To unset the stylesheet and return to a light theme just call:

getApp()->setStyleSheet( "" );

Alternatively you can do it by changing the application palette.

Windows helper functions:

bool windowsDarkThemeAvailable()
{
    // dark mode supported Windows 10 1809 10.0.17763 onward
    // https://stackoverflow.com/questions/53501268/win10-dark-theme-how-to-use-in-winapi
    if ( QOperatingSystemVersion::current().majorVersion() == 10 )
    {
        return QOperatingSystemVersion::current().microVersion() >= 17763;
    }
    else if ( QOperatingSystemVersion::current().majorVersion() > 10 )
    {
        return true;
    }
    else
    {
        return false;
    }
}

bool windowsIsInDarkTheme()
{
    QSettings settings( "HKEY_CURRENT_USER\\Software\\Microsoft\\Windows\\CurrentVersion\\Themes\\Personalize", QSettings::NativeFormat );
    return settings.value( "AppsUseLightTheme", 1 ).toInt() == 0;
}

Currently there seems to be no way to conect to a signal or event that shows the theme has changed in Windows. So I connected to a signal from a QTimer that fires every 5 seconds to check windowsIsInDarkTheme().

Icons

When the theme changes you potentially need to update any icons you have set, e.g. for the toolbar.

In a light theme you can usually set the active icons and let Qt calculate the corresponding disabled icons. This doesn’t work for a dark theme as you want the disabled icons to be darker than the enabled icons, rather than lighter. So you can either calculate the disabled icons programmatically or you can provide a set of disabled icons as well. I opted for the former.

Assuming your icons are set up as resources under :/icons/dark and :/icons/light you can do something like this:

QString getResourceName( const QString& iconName, bool dark )
{
    return QString( ":/icons/%1/%2" ).arg( dark ? "dark" : "light" ).arg( iconName );
}

QPixmap getPixmapResource( const QString& iconName, bool dark )
{
    QString resourceName = getResourceName( iconName, dark );
    QPixmap pixmap = QPixmap( resourceName );
    Q_ASSERT( !pixmap.isNull() );
    return pixmap;
}

QIcon getIconResource( const QString& iconName, bool dark )
{
    QIcon icon;
    QPixmap pixmap = getPixmapResource( iconName, dark );
    icon.addPixmap( pixmap );
    if ( dark )
    {
        // automatic disabled icon is no good for dark
        // paint transparent black to get disabled look
        QPainter p( &pixmap );
        p.fillRect( pixmap.rect(), QColor( 48, 47, 47, 128 ) );
        icon.addPixmap( pixmap, QIcon::Disabled );
    }
    return icon;
}

Then you can reset the icon for the appropriate theme with:

bool isDark()
{
#ifdef Q_OS_MACX
   return macIsInDarkTheme();
#else
   return windowsIsInDarkTheme();
#endif
}
...
myButton->setIcon( getIconResource( "my_icon.png", isDark() ) );
...

You may also be able to update icons through QIcon::setThemeName(). But I didn’t explore this in any detail.

Note that you probably don’t want the enabled icons to be pure white, as it is a bit too visually jarring against a dark theme.

Adding colour schemes to Easy Data Transform

Easy Data Transform user interface.

The colours used in Easy Data Transform make no difference to the output. But the colours are an important part of a user interface, especially when you using a tool for significant amounts of time. First impressions of the user interface are also important from a commercial point of view.

But colour is a very personal thing. Some people are colour-blind. Some people prefer light palettes and others dark palettes. Some people like lots of contrast and other don’t. So I am going to allow the user to fully customize the Center pane colours in Easy Data Transform.

I also want to include some standard colour schemes, to get people started. Looking around at other software it seems that the ‘modern’ trend is for pastel colours, invisible borders and subtle shadows. This looks lovely, but it is a bit low contrast for my tired old eyes. So I have tried to create a range of designs in that hope that everyone will like at least one. Below are the standard schemes I have come up with so far. They all stick with the convention pink=input, blue=transform, green=output.

Which is your favourite (click the images to enlarge).

Is there a tool that you use day to day that has particular nice colour scheme?

I hope to also add an optional dark theme for the rest of the UI in due course (Qt allowing).

Running Qt apps on M1 ARM Macs

Apple is switching the processor architecture of it’s Macs. Again (I transitioned PerfectTablePlan from PowerPC to Intel some hears ago). This time to their own M1 ARM chips. Reports so far have been very positive about speed and battery life of the new processors. Obviously most current Mac software has been written for Intel Macs, so they are using the Rosetta2 emulation layer to run apps compiled for Intel Macs on the ARM chips. I’m not sure how much of a performance hit this causes, but clearly it would be better to run native ARM binaries on an ARM machine. Also Apple, being Apple, want to move everyone to ARM as quickly as possible. Tough luck if you just spent big bucks on a shiny new Intel Mac.

One of my customers emailed me that the latest version of my Hyper Plan visual planner, built with Qt 5.13.1, didn’t run on an new M1 Mac. I don’t currently have an M1 Mac to test it on. But my Easy Data Transform software , built with Qt 5.15.2, apparently works fine on an M1 Mac. So I recompiled Hyper Plan using Qt 5.15.2, and was told it now works. I have found a couple of minor differences in behaviour between Qt 5.13.1 and 5.15.2, but they are too obscure to go into here. Some Qt apps may still have issues on ARM.

Currently Qt is only available as Intel binaries. Efforts are in progress to be able to build Qt as M1 (ARM) binaries. When that is complete it should be possible to ship Qt applications as a ‘fat binary’ with both Intel and ARM executables, as I did with the PowerPC to Intel transition. I’m not sure if this is going to be supported on Qt 5 and 6 or just Qt 6.

** Update Dec-2021 **

Qt 6.2 supports building M1 ARM and Intel binaries. There is no official support for M1 Arm binaries for Qt 5.

** Update Apr-2022 **

Evan of ModernCSV alterted me to an article on deploying a ‘fat binary’ from Qt 5.

Issues with Qt applications on macOS 11.0 (Big Sur)

In my previous post I wrote about the trials and tribulations of upgrading my iMac to macOS 11.0. Here I am going to list some of the issues I know about deploying Qt applications on macOS 11.0. More issues may subsequently come to light.

The QFileDialog::DontConfirmOverwrite flag is ignored when passed to QFileDialog::getSaveFileName(). Which means that you can’t use this flag and handle the message yourself, or you will end up with a double warning. This seems to have been an issue since macOS 10.15. It still isn’t fixed in Qt 5.15.2. It is annoying, but relatively easy to work around. The Qt bug report is QTBUG-39791.

QMessageBox::information() shows a placeholder icon instead of the information icon:

It is only cosmetic. But it looks shonky and Mac users tend to care a lot about this sort of thing. I can reproduce it in Qt 5.15.2. I don’t know of a workaround. The Qt bug report is QTBUG-88928.

QMessageBox::warning(), QMessageBox::information() etc show the default focus button incorrectly. For example:

QMessageBox::warning( this, "App", "text", QMessageBox::Ok|QMessageBox::Cancel, QMessageBox::Ok );

Gives:

Again it is only cosmetic, but it looks jarring. I can reproduce it in Qt 5.15.2. I don’t know of a workaround. The Qt bug report is QTBUG-89133.

There are also some other styling issues. The Qt bug report is QTBUG-86513.

Dark Mode still doesn’t work properly for Qt apps.

There was an issue on macOS 10.15 where using QFileDialog::getSaveFileName() to save over an existing file could cause a crash. Thankfully that doesn’t seem to be an issue in macOS 10.11. The Qt bug report is QTBUG-83342.

Unfortunately issues with Qt on Mac are nothing new. I realize it is a big challenge for the Qt developers to keep such a large codebase up-to-date with so many continually evolving platforms. But the Mac version always feels rather neglected compared to the Windows version. I wish they would prioritise basic issues such as the above over adding whizzy new features, 80% of which most Qt developers probably never use. macOS 11.0 was released a couple of weeks ago and betas have been available for a while.

I would be interested to hear of the experience of other developers with macOS 11.0. Any other Qt macOS 11.0 issues I should know about? Please let me know in the comments.

Upgrading to MacOS 11.0 (Big Sur)

It is always a bit of fraught process upgrading a computer OS, especially for a development machine with loads of tools and libraries installed. So I try to do it as infrequently as I can get away with. On Windows I generally buy a new PC rather than upgrade OS. However glitches had been reported in Easy Data Transform on macOS 11.0 (Big Sur) and I wasn’t ready to abandon my 2017 iMac, so I decided to bite the bullet and upgrade it from macOS 10.13 to 11.0.

The initial upgrade of OS was straightforward enough. But when I tried to run Qt Creator the CPU shot to 99% and stayed there, making the machine unusable. A glance at Activity Monitor showed that several XCode related processes were going crazy. After a bit a Googling I managed to find this magic incantation to type into the terminal on a forum post:

defaults write com.apple.dt.Xcode DVTDisableMainThreadChecker 1

I was then able to rebuild my Qt-based products: Easy Data Transform, PerfectTablePlan and Hyper Plan using the existing installs of Qt 5.13.1 and Qt Creator 4.8.0.

I had to update some of the software I use:

  • DropDMG
  • Beyond Compare
  • SnagIt

Annoyingly, I had to buy an upgrade of SnagIt as the 2018 version doesn’t work on Big Sur. Even more annoyingly the upgrade costs nearly as much as a new licence, which feels predatory.

The Subversion command line no longer worked from the terminal, but that was easily fixed by adding /Applications/XCode.app/Developer/usr/bin to PATH in my .profile.

So far I haven’t been able to get the following to work:

  • XCode
  • Hammer4Mac

XCode 10.1 falls over if I try to start it. It says that it requires additional components and then fails to install them. I may upgrade XCode at some point. But I only use the compiler from the command line via QtCreator, so it doesn’t really matter at present.

Hammer4Mac is a static website builder I use to build the PerfectTablePlan website and a couple of other mini sites. I upgraded to the latest version. It starts, but returns ‘Build failed’ for all 3 websites. No clue as to why. I Tweeted the creator, but got no reply. It appears to be abandonware. If so they should really take down the Hammer4Mac website. I guess I will use it from my macOS 10.14 laptop and then eventually do the tedious job of porting those websites to Jekyll.

Hopefully I won’t have to do another major upgrade of macOS any time soon (I may buy a new Mac next time).

Bloviate

I wondered what it would look like if you took a body of text and then used it to generate new text, using Markov chains of different lengths. So I knocked up  quick program to try it.  ‘Bloviate’.

bloviate

Bloviate analyses your source text to find every sequence of N characters and then works out the frequency of characters that come next.

For example, if you set N=3 and your source text contains the following character sequences staring with ‘the’:

‘the ‘, ‘then’, ‘they’, ‘the ‘

Then ‘the’ should be followed 50% of the time by a space, 25% of the time by an ‘n’ and 25% of the time by a ‘y’.

Bloviate then creates output text, starting with the first N characters of the source text and filling in the rest randomly using the same sequence frequencies as the source text.

Note that a character is a character to Bloviate. It treats upper and lower case as different characters, makes no attempt to differentiate between letters, punctuation and white space and does not attempt to clean up the source text. Which also means it works on any language.

Bloviate also tells you the average number of different characters following each unique sequence of N, which I will call F here. As F approaches 1.0 the output text becomes closer and closer to the input text.

Using ‘Goldilocks and the 3 bears’ as input:

If N=1 (F=7.05) the output is garbage. Albeit garbage with the same character pair frequency as the original.

On cre She sl s ramy raked cheais Bus ore than s sherd up m. ged. bend staireomest p!”Sof ckstirigrorr a ry ps.

” f waine tind s aso Sowa t antthee aime bupis stht stooomed pie k is beche p!

At N=3 (F=1.44) it looks close to English, but jibberish:

Once up and been sight,” she this timed. Pretty so soon, she second soft. She screame up and she screame hot!” cried the Mama bed the Papa been sleeping in the Papa bear

“Someone’s bear growl.

At N=5 (F=1.14) it starts to look like proper English, but semantically weird:

Once upon a time, so she went for a walked right,” she lay down into the kitchen, Goldilocks sat in the porridge from the three chair,” growled, “Someone’s been sitting my porridge and she tasted the door, and ran down the bedroom. Goldilocks woke up and she second bowl.

And it comes out with occasional gems such as:

“Someone’s been sitting my porridge,” said the bedroom.

At N=10 (F=1.03) it starts to become reasonably coherent:

Once upon a time, there was a little tired. So, she walked into the forest. Pretty soon, she came upon a house. She knocked and, when no one answered, she walked right in.

At the table in the kitchen, there were three bowls of porridge.

At N=15 (F=1.01) it starts to get pretty close to the original text, but doesn’t follow quite the same order:

Once upon a time, there was a little girl named Goldilocks. She went for a walk in the forest. Pretty soon, she came upon a house. She knocked and, when no one answered, she walked right in.

At the table in the kitchen, there were three bowls of porridge. Goldilocks was very tired by this time, so she went upstairs to the bedroom. She lay down in the first bed, but it was too hard. Then she lay down in the third bed and it was just right. Goldilocks fell asleep.

At N=12 (F=1.07) the whole 680k characters of ‘Pride and prejudice’ produces:

It is a truth universally contradict it. Besides, there was a motive within her of goodwill which could not help saying:

“Oh, that my dear mother had more command over herself! She can have her own way.”

As she spoke she observed him looking at her earnest desire for their folly or their vice. He was fond of them.”

Obviously the source text is important. The Bohemian Rhapsody lyrics make nearly as much (or as little sense) at N=5 (F=1.08) as the original:

Is this to me, for me, to me

Mama, just a poor boy from this to me

Any way the truth

Mama, life? Is this time tomorrow

Carry on as if nothing all behind and face the truth

Mama, ooh, didn’t mean to me, baby!

Just gotta leave me and lightning, very fright out, just killed a man

Put a gun against his head

Pulled my time to die?

At N=12 (F=1.05) 160k characters of Trump election speeches produces:

Hillary brought death and disaster to Iraq, Syria and Libya, she empowered Iran, and she unleashed ISIS. Now she wants to raise your taxes very substantially. Highest taxed nation in the world is a tenant of mine in Manhattan, so many great people. These are people that have been stolen, stolen by either very stupid politicians ask me the question, how are you going to get rid of all the emails?” “Yes, ma’am, they’re gonna stay in this country blind. My contract with the American voter begins with a plan to end government that will not protect its people is a government corruption at the State Department of Justice is trying as hard as they can to protect religious liberty;

Supply your own joke.

I knocked together Bloviate in C++/Qt in a couple of hours, so it is far from commercial quality. But it is fairly robust, runs on Windows and Mac and can rewrite the whole of ‘Pride and prejudice’ in a few seconds. The core of Bloviate is just a map of the frequency of characters mapped to the character sequence they follow:

QMap< QString, QMap< QChar, int > >

You can get the Windows binaries here (~8MB, should work from Windows 7 onwards).

You can get the Mac binaries here (~11MB, should work from macOS 10.12 onwards).

Note that the Bloviate executable is tiny compared to the Qt library files. I could have tried to reduce the size of the downloads, but I didn’t.

To use Bloviate just:

  1. paste your source text in the left pane
  2. set the sequence length
  3. press the ‘Go >’ button

I included some source text files in the downloads.

You can get the source for Bloviate here (~1MB).

It should build on Qt 4 or 5 and is licensed as creative commons. If you modify it, just give me an attribution and send me a link to anything interesting you come up with.

How to notarize your software on macOS

Apple now wants you to ‘notarize’ your software. This is a process where you upload your software to Apple’s server so it can be scanned and certified malware free. This will probably become compulsory at some point, even (especially?) if your software isn’t in the Apple app store. Apple says:

Give users even more confidence in your software by submitting it to Apple to be notarized. The service automatically scans your Developer ID-signed software and performs security checks. When it’s ready to export for distribution, a ticket is attached to your software to let Gatekeeper know it’s been notarized.

When users on macOS Mojave first open a notarized app, installer package, or disk image, they’ll see a more streamlined Gatekeeper dialog and have confidence that it is not known malware.

Note that in an upcoming release of macOS, Gatekeeper will require Developer ID signed software to be notarized by Apple.

Documentation on notarization is a bit thin on the ground, especially if you want to notarize software that wasn’t built using XCode (I build my software using QtCreator). So I am writing up my experiences here.

First you need to ensure you have macOS 10.14 and XCode 10 installed (with command line tools) and you need a current Apple developer account.

Codesign your app with ‘hardened runtime’ using --options runtime :

codesign --deep --force --verify --verbose --sign "Developer ID Application:<developer id>" --options runtime <app file>

E.g.:

codesign --deep --force --verify --verbose --sign "Developer ID Application: Acme Ltd" --options runtime myApp.app

A ‘hardened runtime’ limits the data and resourced an application can access. I’m not sure what the exact ramification of this are. But it doesn’t seem to have restrict my software from doing anything it could do previously.

You can check the signing with:

codesign --verify --verbose=4 <app file>

E.g.:

codesign --verify --verbose=4 myApp.app

Now package your app into a .dmg (e.g. using DropDMG). Then upload the .dmg to Apple’s servers:

xcrun altool -t osx -f <dmg file> --primary-bundle-id <bundle id> --notarize-app --username <username>

E.g.:

xcrun altool -t osx -f myApp.dmg --primary-bundle-id com.acme.myapp --notarize-app --username me@acme.com

You will be prompted for your Apple developer password (or you can include it on the command line).

You now have to wait a few minutes. If the upload is successful “No errors uploading ” will be shown and a unique ID will be returned. You then have to use this to request your upload be scanned:

xcrun altool --notarization-info <notarize ID> -u <username>

E.g.:

xcrun altool --notarization-info xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx -u me@acme.com

You will be prompted for your Apple developer password (or you can include it on the command line).

Hopefully you will see “Status Message: Package Approved”. If the notarization fails, you should be sent a link to an online log file describing the issue. If the notarization completes successfully you need to ‘staple’ the results to your .dmg:

xcrun stapler staple -v <dmg file>

E.g.:

xcrun stapler staple -v myApp.dmg

The stapler outputs a log including some odd phrases. Mine included: “Humanity must endure”, “Let’s see how that works out. “, “Adding 1 blobs to superblob. What about Blob?” and “Enjoy”. Weird. Hopefully it will end with “The staple and validate action worked!”.

Finally you can unpack your .dmg into a .app and verify it with:

spctl -a -v <app file>

E.g.

spctl -a -v /Applications/myApp.app

On macOS 10.14 (but not earlier OSs) it should say “source=Notarized Developer ID”. Your software should now run on 10.14 without a warning dialog. Congratulations!

It all seems rather clumsy. As you have to wait asynchronously for the unique ID to be returned from step 1 before you can complete step 2, it is not easy to fully automate in a script. This is a major pain the arse. If anyone works out a way to automate it the whole process, please let me know.

Here are some links to the various posts that I gleaned this information from:

https://cycling74.com/forums/apple-notarizing-for-mojave-10-14-and-beyond
https://www.mbsplugins.de/archive/2018-11-02/Notarize_apps_for_MacOS
https://forum.xojo.com/50655-how-to-codesign-and-notarise-your-app-for-macos-10-14-and-highe
https://forum.xojo.com/49408-10-14-hardened-runtime-and-app-notarization/11
https://stackoverflow.com/questions/53112078/how-to-upload-dmg-file-for-notarization-in-xcode
https://lapcatsoftware.com/articles/debugging-mojave.html

** Update 28-Apr-2023 **

altool is now deprecated. See Moving from altool to notarytool for Mac notarization for an update.

Qt is broken on macOS right now

Let me say up front that I am a big fan of the Qt framework. I have been working with it continuously since the late 90s. Both of my commercial products (PerfectTablePlan and Hyper Plan) are written on top of Qt. But Qt is quite broken on macOS right now. In fact I struggled to find a version of Qt 5 that supported recent versions of macOS and didn’t have a showstopper bug. I must have wasted a couple of weeks grappling with these issues. I am putting my notes here in the hope that they help someone else.

Here are the 4 bugs in macOS version of Qt that have causing me the most headaches:

Qt Version
bug v5.9.6 v5.10.1 v5.11.0 v5.11.1 v5.11.2
QFontDialog broken (link) ok ok ok ss ss
QStaticText::size() returns wrong value for rich text
with line breaks
ss ss ok ok ok
widgets such as QTableWidget do not repaint correctly (link) ok cwa cwa cwa cwa
default QComboBox size is incorrect (link)
ok ok cwa ok ok

Where:

ok = bug not present.

cwa = bug present, but can workaround.

ss= bug present, making release unusable for me (showstopper).

Consequently Qt 5.11.0 is the only usable release on macOS for me and I have to kludge my way around two quite nasty bugs. While the QStaticText issue is fairly obscure (but important for me), how in the hell did QFontDialog get completely broken without anyone noticing? And then not get fixed for two whole releases?! I also know other developers who are having to stick with earlier versions of Qt due to the bugginess of the recent versions of Qt on macOS. This isn’t good enough.

Thankfully the recent versions of Qt are much more solid on Windows.

** Feb-2019 Update: Qt 5.12.0 seems to be a big improvement. **

** Mar-2019 Update: I had to revert PerfectTablePlan to Qt 5.11.0 because versions built with 5.12.0 and 5.12.1 kept crashing deep in Qt. **

** Sep-2019 Update: I had to revert PerfectTablePlan to Qt 5.11.0  again because versions built with 5.12.4 kept crashing deep in Qt. **

** Oct-2019 Update: Qt 5.12.5 fixes all the bugs above and doesn’t seem to introduce any major new bugs that I care about. **

Getting Qt 5.9 working on Windows (eventually)

I have had Qt 5.5 and 5.6 installed on my development machines for some time. Now that I have purchased a new Mac development box (an iMac with a lickably beautiful 27″ screen) I thought it was a good time to update to a more recent version of Qt. I went for Qt 5.9, rather than Qt 5.10, as 5.9 has been designated as an LTS (long term support) release. Upgrading turned into a real chore. I am quickly writing it up here in the hope that it helps someone else, and as a reminder to myself a few years down the line.

I like to build Qt from source. Because then I know it was built using the same compiler, headers, SDK etc as I am using to build my product. And I have more control over how Qt is configured. Also I can patch the source and rebuild it, if I need to. But I have had problems building Qt on Mac before. So I decided to install the pre-built binaries on my new Mac. I installed the latest version of XCode and then the Q5.9.4 binaries. This was a couple of big downloads, but it all went pretty smoothly.

I successfully built Qt 5.5 from source on my Windows machine previously, so I decided to try that for Qt 5.9. I have Visual Studio 2010 installed. This isn’t supported for Qt 5.9.4, so I downloaded Visual Studio 2017. I unzipped the Qt source into C:\Qt\5.9.4, ran ‘x86 native tools command prompt for VS 2017’, made sure Python and Perl were in the path and then:

cd C:\Qt\5.9.4

set QTDIR=C:\Qt\5.9.4\qtbase

set PATH=%QTDIR%\bin;%PATH%

configure -opensource -confirm-license -opengl desktop -nomake tests -nomake examples -no-plugin-manifests -debug-and-release -platform win32-msvc -verbose

nmake

Note that you are told by the nmake script to do nmake install at the end of this. But it tells you somewhere in the Qt Windows documentation not to do this, unless you have set the prefix argument (confusing, I know)

The build failed part way through making qtwebengine. Something to do with a path being too long for Perl or Python (I forget). It seems to be a known problem. Odd as the root path was just C:\Qt\5.9.4. I don’t need qtwebengine at present, so I deleted everything and tried again with -skip qtwebengine:

configure -opensource -confirm-license -opengl desktop -skip qtwebengine -nomake tests -nomake examples -no-plugin-manifests -debug-and-release -platform win32-msvc -verbose

nmake

It seemed to complete ok this time. But using this version of Qt to build Hyper Plan I got an error:

Unknown module(s) in QT:svg

On further examination the SVG DLL  had been built, but hadn’t been copied to the C:\Qt\5.9.4\qtbase\bin folder. Similarly for a lot of the other Qt DLLs. I couldn’t find any obvious reason for this looking through logs, Stackoverflow and Googling. I could possibly do without the SVG functionality, but I wasn’t sure what else was broken. So I decided to give up on bulding from source on Windows as well.

I download the Qt 5.9.4 binaries for Visual Studio 2017. This seemed to go ok, but then I discovered that I could only build a 64-bit application from these. No 32-bit version was available for Visual Studio 2017. Many of my customers are still on 32 bit versions of Windows. So I need to be able to ship my product as a 32 bit executable + DLLs[1].

So I uninstalled Visual Studio 2017 and installed Visual Studio 2015. I then got an error message about Visual Studio 2017 redistributables that I hadn’t uninstalled. So I had to uninstall those and run a repair install on Visual Studio 2015. That seemed to work ok. So then I download the 32-bit Qt 5.9.4 binaries for Visual Studio 2015. I had to download these into a different top level folder (C:\Qtb), so as not to risk wiping existing Qt installs that I had previously managed to build from source.

Eventually I managed to build Hyper Plan and PerfectTablePlan on Mac and Windows. What a palaver though! Qt is an amazing framework and I am very grateful for everyone who works on it. But I wish they would make it a bit easier to install and upgrade! Has anyone actually managed to get Qt 5.9 built from source on Windows?

[1] I don’t bother shipping a 64-bit executable on Windows as the 32-bit executable works fine on 64-bit versions of Windows (my software doesn’t require excessive amounts of memory). I only ship a 64-bit executable on macOS as almost no-one uses 32-bit versions of macOS now.