Category Archives: qt

Getting Qt 5.9 working on Windows (eventually)

I have had Qt 5.5 and 5.6 installed on my development machines for some time. Now that I have purchased a new Mac development box (an iMac with a lickably beautiful 27″ screen) I thought it was a good time to update to a more recent version of Qt. I went for Qt 5.9, rather than Qt 5.10, as 5.9 has been designated as an LTS (long term support) release. Upgrading turned into a real chore. I am quickly writing it up here in the hope that it helps someone else, and as a reminder to myself a few years down the line.

I like to build Qt from source. Because then I know it was built using the same compiler, headers, SDK etc as I am using to build my product. And I have more control over how Qt is configured. Also I can patch the source and rebuild it, if I need to. But I have had problems building Qt on Mac before. So I decided to install the pre-built binaries on my new Mac. I installed the latest version of XCode and then the Q5.9.4 binaries. This was a couple of big downloads, but it all went pretty smoothly.

I successfully built Qt 5.5 from source on my Windows machine previously, so I decided to try that for Qt 5.9. I have Visual Studio 2010 installed. This isn’t supported for Qt 5.9.4, so I downloaded Visual Studio 2017. I unzipped the Qt source into C:\Qt\5.9.4, ran ‘x86 native tools command prompt for VS 2017’, made sure Python and Perl were in the path and then:

cd C:\Qt\5.9.4

set QTDIR=C:\Qt\5.9.4\qtbase

set PATH=%QTDIR%\bin;%PATH%

configure -opensource -confirm-license -opengl desktop -nomake tests -nomake examples -no-plugin-manifests -debug-and-release -platform win32-msvc -verbose

nmake

Note that you are told by the nmake script to do nmake install at the end of this. But it tells you somewhere in the Qt Windows documentation not to do this, unless you have set the prefix argument (confusing, I know)

The build failed part way through making qtwebengine. Something to do with a path being too long for Perl or Python (I forget). It seems to be a known problem. Odd as the root path was just C:\Qt\5.9.4. I don’t need qtwebengine at present, so I deleted everything and tried again with -skip qtwebengine:

configure -opensource -confirm-license -opengl desktop -skip qtwebengine -nomake tests -nomake examples -no-plugin-manifests -debug-and-release -platform win32-msvc -verbose

nmake

It seemed to complete ok this time. But using this version of Qt to build Hyper Plan I got an error:

Unknown module(s) in QT:svg

On further examination the SVG DLL  had been built, but hadn’t been copied to the C:\Qt\5.9.4\qtbase\bin folder. Similarly for a lot of the other Qt DLLs. I couldn’t find any obvious reason for this looking through logs, Stackoverflow and Googling. I could possibly do without the SVG functionality, but I wasn’t sure what else was broken. So I decided to give up on bulding from source on Windows as well.

I download the Qt 5.9.4 binaries for Visual Studio 2017. This seemed to go ok, but then I discovered that I could only build a 64-bit application from these. No 32-bit version was available for Visual Studio 2017. Many of my customers are still on 32 bit versions of Windows. So I need to be able to ship my product as a 32 bit executable + DLLs[1].

So I uninstalled Visual Studio 2017 and installed Visual Studio 2015. I then got an error message about Visual Studio 2017 redistributables that I hadn’t uninstalled. So I had to uninstall those and run a repair install on Visual Studio 2015. That seemed to work ok. So then I download the 32-bit Qt 5.9.4 binaries for Visual Studio 2015. I had to download these into a different top level folder (C:\Qtb), so as not to risk wiping existing Qt installs that I had previously managed to build from source.

Eventually I managed to build Hyper Plan and PerfectTablePlan on Mac and Windows. What a palaver though! Qt is an amazing framework and I am very grateful for everyone who works on it. But I wish they would make it a bit easier to install and upgrade! Has anyone actually managed to get Qt 5.9 built from source on Windows?

[1] I don’t bother shipping a 64-bit executable on Windows as the 32-bit executable works fine on 64-bit versions of Windows (my software doesn’t require excessive amounts of memory). I only ship a 64-bit executable on macOS as almost no-one uses 32-bit versions of macOS now.

Pretty printing C++ with Clang-Format

I use some of the code generation and refactoring tools in QtCreator. These save a lot of time, but they don’t format C++ code how I like it. For example they produce C++ code like this:

void MyClass::foo(int *x)

But I like my code formatted like this:

void MyClass::foo( int* x )

The differences may seem minor, but they are a source of significant irritation to me. I like my code how I like it, goddammit! And consistent formatting enhances readability. However re-formatting it by hand is time-consuming and tedious.

What I need is a tool that can enforce consistent formatting in the style that I like, or something close. I have tried to use automatic C++ formatting (pretty printing) tools in the past, but I couldn’t get them to produce a format that was close enough to what I wanted. But I have finally found the tool for the job. Clang-Format.

Clang-Format is part of the LLVM family of tools. It is a free, command-line tool that reformats C++, Objective-C or C according to the settings in a config file. As with many free tools, it isn’t terribly well documented. Some of the documentation on the web is out of date and some of it is incomplete. But I have managed to find out enough to configure it how I like it.

To run it you just need to place your options in a .clang-format file, make sure the clang-format executable is in the path and then run it:

clang-format.exe -i -style=file <C++ file>

Here are the settings I am currently using in my .clang-format file:

Language: Cpp
AccessModifierOffset: -4
AlignAfterOpenBracket: false
AlignConsecutiveAssignments: false
AlignConsecutiveDeclarations: false
AlignEscapedNewlinesLeft: false
AlignOperands: true
AlignTrailingComments: false
AllowAllParametersOfDeclarationOnNextLine: false
AllowShortBlocksOnASingleLine: false
AllowShortCaseLabelsOnASingleLine: false
AllowShortFunctionsOnASingleLine: Inline
AllowShortIfStatementsOnASingleLine: false
AllowShortLoopsOnASingleLine: false
AlwaysBreakAfterDefinitionReturnType: None
AlwaysBreakAfterReturnType: None
AlwaysBreakBeforeMultilineStrings: false
AlwaysBreakTemplateDeclarations: false
BinPackArguments: true
BinPackParameters: true
BraceWrapping:
  AfterClass:      true
  AfterControlStatement: true
  AfterEnum:       true
  AfterFunction:   true
  AfterNamespace:  true
  AfterObjCDeclaration: true
  AfterStruct:     true
  AfterUnion:      false
  BeforeCatch:     true
  BeforeElse:      true
  IndentBraces:    false
BreakBeforeBinaryOperators: None
BreakBeforeBraces: Allman
BreakBeforeTernaryOperators: true
BreakConstructorInitializersBeforeComma: false
ColumnLimit: 0
CommentPragmas: '^ IWYU pragma:'
ConstructorInitializerAllOnOneLineOrOnePerLine: false
ConstructorInitializerIndentWidth: 0
ContinuationIndentWidth: 4
Cpp11BracedListStyle: true
DerivePointerAlignment: false
DisableFormat: false
ExperimentalAutoDetectBinPacking: false
ForEachMacros: [ foreach, Q_FOREACH, BOOST_FOREACH ]
IndentCaseLabels: true
IndentWidth: 4
IndentWrappedFunctionNames: false
KeepEmptyLinesAtTheStartOfBlocks: true
MacroBlockBegin: ''
MacroBlockEnd: ''
MaxEmptyLinesToKeep: 2
NamespaceIndentation: None
PenaltyBreakBeforeFirstCallParameter: 100
PenaltyBreakComment: 300
PenaltyBreakFirstLessLess: 120
PenaltyBreakString: 1000
PenaltyExcessCharacter: 10000
PointerAlignment: Left
ReflowComments: true
SortIncludes: false
SpaceAfterCStyleCast: false
SpaceBeforeAssignmentOperators: true
SpaceBeforeParens: ControlStatements
SpaceInEmptyParentheses: false
SpacesBeforeTrailingComments: 1
SpacesInAngles: true
SpacesInContainerLiterals: true
SpacesInCStyleCastParentheses: true
SpacesInParentheses: true
SpacesInSquareBrackets: true
Standard: Cpp11
TabWidth: 4
UseTab: Never

It took me a few hours of fiddling with the settings to find the best combination. It would be really useful if someone could write a tool that would analyze your C++ code and create a .clang-format file for you. You would probably only want to do this once though, so I don’t think it has much potential as a commercial product.

There are only two things I couldn’t get quite right in the formatting:

  1. I couldn’t get it to add a blank line after public, protected and private declarations. I fixed this with a quick Perl hack (see below).
  2. I couldn’t get it to indent continuation lines how I would like (ideally indented 1 or 2 spaces from the first line). It is a small price to pay and I am just putting up with it for now.

Perhaps there are options to do these and I just didn’t find them.

Here is the Windows .bat script I used to format all the C++ files in a folder.

for %%f in (*.h *.cpp *.inl) do (
clang-format.exe -i -style=file %%f
)

for %%f in (*.h) do (
clang-format.exe -i -style=file %%f
perl -p -i.bak -e "s/public:/public:\n/g" %%f
perl -p -i.bak -e "s/protected:/protected:\n/g" %%f
perl -p -i.bak -e "s/private:/private:\n/g" %%f
perl -p -i.bak -e "s/    Q_OBJECT/Q_OBJECT/g" %%f
)

del *.bak
del *.tmp

No doubt there is a more elegant way to do the Perl, but it works.

I now just run this batch periodically to keep my code beautiful and consistent.

Signing Qt applications for Mac OS X 10.9.5 and 10.10

I have written previously about signing Qt applications for Mac OS X. It all worked fine until I upgraded to Mac OS X 10.9.5, which broke my signing script. Those Apple chaps do love to break stuff. Grrr.

The problem appears to be that the directory structure of the app bundle has changed and the Qt4 macdeployqt command does not conform to the new layout (I believe this is also the case for Qt5). Oh joy. I managed to work out how to get it working again after a bit of digging around. The good news is that Apple have also made the codesign command easier with a --deep option to traverse and sign the whole bundle in a single command. About time.

So here is the basic process to build and sign your Qt .app on the latest versions of Mac OS X:

# deploy Qt frameworks into .app bundle
$QTDIR/bin/macdeployqt <your_app>.app -verbose=1
# optionally delete unwanted framework and plugin folders, e.g.:
# rm -f -r <your_app>.app/Contents/Frameworks/QtDeclarative.framework
# rm -f -r <your_app>.app/Contents/PlugIns/sqldrivers
# correct .app bundle structure
python rebundle.py $QTDIR <your_app>.app
# sign .app bundle (including frameworks and plugins)
codesign --deep --force --verify --verbose --sign "Developer ID Application: <your developer id>" <your_app>.app
# the 2 lines below are just for verification/diagnostics
otool -L <your_app>.app/Contents/MacOS/<your_app>
codesign --verify --verbose=4 <your_app>.app

(Sorry about the small font, but I wanted to avoid confusing line wraps).

I then invoke DropDmg to create a .dmg image file complete with licence and background image. This is all stuck it all in a bash script, which I can pretty much forget about it (until Apple break something else).

In the above rebundle.py is a Python script  written by some public spirited individual that can be downloaded from github (thank you, ‘kingcheez’). Note that you can just find and replace all the ‘5’ characters in the script by ‘4’ if you are still using Qt4.

The first time I ran my script I ended up with a whopping 50MB .app file. It turns out that the cp -r commands in my script don’t preserve symbolic links. So you end up with 3 copies of each framework library. You can avoid this by using cp -R instead.

On the subject of signing for Mac, Apple recently sent out an email stating:

Signatures created with OS X Mountain Lion 10.8.5 or earlier (v1 signatures) will be obsoleted and Gatekeeper will no longer recognize them. Users may receive a Gatekeeper warning and will need to exempt your app to continue using it. To ensure your apps will run without warning on updated versions of OS X, they must be signed on OS X Mavericks 10.9 or later (v2 signatures). … Apps signed with v2 signatures will work on older versions of OS X.

So you are going to have to start signing using 10.9, whether you like it or not.

How to build Qt 4.8.5 on Mac OS X 10.9

I prefer to build Qt from source. I have been trying to build Qt 4.8.5 on Mac OS X 10.9 (Mavericks). I managed in the end, but it took a few tweaks. Online information about how to do this was fragmentary, so I am documenting it here in case it is useful to someone else.

1. Webkit doesn’t build. I don’t need it so I disabled it using configure option:

-no-webkit

2. The corewlan plugin doesn’t build. I don’t need it (I think, I’ve never heard of it before) so I disabled it using this fix from stackoverflow.

3. The TIFF image format plugin doesn’t build. I don’t need it so I disabled it using configure option:

-no-libtiff

My final configure command was:

./configure -nomake demos -nomake examples -debug-and-release -no-multimedia -no-audio-backend -no-phonon -no-phonon-backend -no-gif -no-openssl -no-webkit -no-libtiff

There will be lots of warnings that Qt 4.8.5 isn’t compatible with Mac OS X 10.9. But you can ignore these (or comment out the warning in the appropriate Qt header file).

I have done some brief experiments and it seems to work ok. Hopefully there will be a Qt 4.8.6 that fixes these issues. Note that you also need to make some tweaks to your application code. See:

Fixing Qt 4 for Mac OS X 10.9

** UPDATE April-2014 **

Qt 4.8.6 has been released. This appears to build fine on Mac OS X 10.9.

Fixing Qt 4 for Mac OS X 10.9

Mac OS X 10.9 (Mavericks) was released yesterday. And those nice people at Apple made it free, so you can be sure lots of people are downloading it. However Qt 4 apps look at bit strange on the new OS. Look at the text alignment in these buttons:

buttons1The text isn’t centre aligned. It doesn’t look like much of an issue out of context. But it looks wrong when you look at a whole UI. The good news is that there is a simple fix:

#ifdef Q_OS_MACX
    if ( QSysInfo::MacintoshVersion > QSysInfo::MV_10_8 )
    {
        // fix Mac OS X 10.9 (mavericks) font issue
        // https://bugreports.qt-project.org/browse/QTBUG-32789
        QFont::insertSubstitution(".Lucida Grande UI", "Lucida Grande");
    }
#endif

You need to place this code in your main() before creating your QApplication. For more details see this bug report.

With the fix the buttons look like this:

buttons2Much better! There are some console warnings:

CoreText performance note: Client called CTFontCreateWithName() using
name "Lucida Grande" and got font with PostScript name "LucidaGrande". 
For best performance, only use PostScript names when calling this API.

I am not sure how how significant these are.

I have also found that updating to Qt 4.8.5 fixes a printing crash bug in my table plan software. This crash happened when rotated pixmaps were printed from Mac OS X 10.8.

I have seen on forums that Qt 5 is completely broken on 10.9. But I don’t know if that is true.

Boostrapped.fm podcast

I was a guest on episode 21 of Bootstrapped.fm, the podcast of Andrey Butov and Ian Landsman. The discussion was very wide-ranging, touching on SAAS vs web, the Qt development environment, the royal wedding, A/B testing, capoeira, Adwords, the history of shareware, my new training course and lots more besides. I really enjoyed it. Boostrapped.fm also has a thriving discussion forum at discuss.bootstrapped.fm.

How to sign your Mac OS X App for Gatekeeper

If a prospective customer downloads your software onto Mac OS X 10.8 and it hasn’t been signed, they will see a scary warning:

Not good. To run unsigned software they need to go into Mac OS X Preferences>Security & Privacy>General and change Allow applications downloaded from Mac App store and identified developers to Anywhere:

Or they need to right/Ctrl click and see another scary warning. Double plus not good. This is the new Mac Gatekeeper system in action. Apple being Apple, Gatekeeper defaults to only allowing users to run software they have downloaded off the Internet if it has been signed. This could have a big effect on your conversion rate on Mac. So if you are shipping software for the Mac, you really need to sign it.

Apple fanboys will tell this is a sensible way for Apple to control software quality. A valid certificate shows that your software hasn’t been tampered with and, if it turns out to be malware, Apple can revoke your certificate. The more cynical might see it as a way for Apple to exert even greater control over Mac developers than it already does, while simultaneously extorting $99 per year from each and every one of them. Make your own mind up on that one.

I have now managed to sign my table planner software, ready for its next release. I should have done it months ago. But I expected the process to be so tedious that it has taken me this long to get around to it. And it was every bit as mind-numbingly tedious as I expected trying to find a few useful nuggets amongst the acres of Apple documentation. I found some useful stuff in blogs, but it was quite fragmented. So I have thrown together these notes in the hope that it saves someone else a few hours going round in circles. Note that I am not currently submitting my software to the Mac App Store, so I don’t cover that here. Also my software is developed in C++/Qt using Qt Creator, rather than Objective-C/Cocoa using XCode, and my approach reflects that.

1. Sign up for Apple Developer Connection ($99 per year). Doesn’t matter if you already paid through the nose for a Windows authenticode certificate. Gatekeeper only accepts Apple certificates, so you have no choice. On the plus side, you do get other benefits, including downloading new OS upgrades for free.

2. You need Mac OS X 10.8 so you can test that your signing works. If you have an Apple Developer Connection subscription, you can download 10.8 for free (get a code from the ADC downloads area and using it in the Mac App Store). I found the upgrade from 10.6 to 10.8 was surprisingly painless (Microsoft eat your heart out).

3. Request your Apple certificates and install them into your Keychain. You can do this from Xcode (instructions here). You may need to upgrade Xcode to a recent version.

4. Use the codesign command line tool to sign:

  • Every framework in your .app bundle
  • Every plugin in your .app bundle
  • Your .app file

I believe you can do this as part of your Xcode build. But I prefer a shell script. For example:

echo --sign frameworks --
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Frameworks/QtCore.framework/Versions/4/QtCore
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Frameworks/QtGui.framework/Versions/4/QtGui
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Frameworks/QtNetwork.framework/Versions/4/QtNetwork
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Frameworks/QtSql.framework/Versions/4/QtSql
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Frameworks/QtXml.framework/Versions/4/QtXml
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Frameworks/Qt3Support.framework/Versions/4/Qt3Support

echo --sign plugins--
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Plugins/accessible/libqtaccessiblecompatwidgets.dylib
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Plugins/accessible/libqtaccessiblewidgets.dylib
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Plugins/bearer/libqcorewlanbearer.dylib
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Plugins/bearer/libqgenericbearer.dylib
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Plugins/codecs/libqcncodecs.dylib
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Plugins/codecs/libqjpcodecs.dylib
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Plugins/codecs/libqkrcodecs.dylib
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Plugins/codecs/libqtwcodecs.dylib
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Plugins/graphicssystems/libqtracegraphicssystem.dylib
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app/Contents/Plugins/imageformats/libqjpeg.dylib

echo --sign app--
codesign --force --verify --verbose --sign "Developer ID Application: <yourID>" <yourApp>.app

I do this in a build shell script that automates the whole process of creating a .dmg for download. I’m not sure if the order you sign the components in is important.

Note that:

  • <yourID> is the ID on your certificate (in my case “Oryx Digital Ltd”).
  • For frameworks you sign the folder, not the file.
  • Any changes to the .app bundle after signing may invalidate the signature (that is kind of the point).

5. Verify the  signing of the .app file. For example:

codesign -vvv -d <yourApp>.app

6. Package your .app into a .dmg, .zip, .pkg or whatever other format you use to install it (I believe .pkg files might require additional signing with a different certificate).

7. Make sure your Mac OS X 10.8 machine is set to the default Gatekeeper setting.

8. Download your software onto Mac OS X 10.8 and check if the scary warning has gone away.

9. Pray that Apple doesn’t decide to revoke your certificate at some point for an infraction, real or imagined.

Until you have released a signed version you can put up a warning with some simple Javascript, for example:

Further reading:

http://www.hardcoded.net/devlogs/20120407

http://www.mactech.com/articles/mactech/Vol.24/24.11/CodeSigning-GetUsedtoIt!/index.html

http://www.macworld.co.uk/macsoftware/news/?newsid=3338078

http://support.apple.com/kb/HT5290

http://www.macworld.com/article/1165408/mountain_lion_hands_on_with_gatekeeper.html

http://developer.apple.com/library/mac/#documentation/ToolsLanguages/Conceptual/OSXWorkflowGuide/CodeSigning/CodeSigning.html

http://developer.apple.com/library/mac/#documentation/ToolsLanguages/Conceptual/OSXWorkflowGuide/DistributingApplicationsOutside/DistributingApplicationsOutside.html

Qt related:

http://lynxline.com/submiting-to-mac-app-store/

http://www.digia.com/en/Blogs/Qt-blog/Pasi_Matilainen/Dates/2012/4/How-to-Publish-Qt-Applications-in-the-Mac-App-Store/

http://comments.gmane.org/gmane.comp.lib.qt.user/637

Java related:

http://blogs.oracle.com/talkingjavadeployment/entry/java_applications_and_gatekeeper

http://www.ej-technologies.com/products/install4j/whatsnew51.html

Thanks to Jonathan of DeepTrawl and Stephane of LandlordMax for some useful pointers.

************** Update **************

Things have changed again for Mac OS X 10.9/10.10. See this post for an update.