A monitor for Cruise Control

I have wanted to build a monitor for our Cruise Control build server ever since I heard about the use of lava lamps to report the build status. The “easy” way seems to be to use an X10 automation system but unfortunately I have not been able to find that for sale in Japan. (It probably conflicts with local radio licensing laws. )

Our builds have been staying broken for tool long. The email notification  does not seem to encourage the fixing of the build. We definately needed something more obvious to show the build status. So when I recently found that there was a 4-way USB relay board available at a fairly reasonable price from Elefine I decided to build my own monitor. The hard part is controlling the power from the computer and the relay board takes care of that. Once I got the board all I needed to do was put it in a case and create some driver software to control it.

Fitting the board into a box and wiring it up was fairly simple. Unfortunately the USB port on the relay board is flush with the edge of the circuit board. This means that a fairly large hole is needed in the case to take the USB plug which looks a bit messy. 

For the lamps I decided not to use lava lamps because they will have to sit on the desk in the office and they tend to run very hot.  Instead I have adapted a small globe light, taking out the normal bulb and replacing it with two separate lights, one red and one green.

For the software I had already made a cruise control monitor plugin for QtCreator so I had most of the code already. All that was needed was to add code to switch the relays on and off depending on the build status. I tried to use the DLL which is provided by emx for the relay board but that did not work well – causing the program to crash on exit. So instead the code uses their command line utility for controlling the board.

Using the 3DConnexion mouse with a Qt Application

This note describes my experiences getting a 3D mouse to work in a Qt application on Windows.

The 3DConnexion 3D mouse is a 6-DOF controller often used for CAD and engineering applications. 3DConnexion provide an SDK for windows which provides integration with MFC applications but not for Qt.


RAWINPUT support for Qt

The recommended method of supporting the 3DConnexion 3D mouse is to use Windows RAWINPUT message to receive the data from the mouse and then process those message to get the motion data.

There are 2 problems here

• Qt has no native support for RAWINPUT - it is a windows only system and Qt has no corresponding Qt events for the 3D mouse
• The "out-of-the-box" version of Qt does not forward RAWINPUT messages as they are received - at least up to Qt 4.6.2. On windows 7 Qt4.7.1 seemed to work with no modification.

The Raw Input API provides a "stable and robust way for applications to accept raw input from any HID, including the keyboard and mouse." [msdn] This enables the application to get the actual data sent by the HID device without any processing being done by windows first.

The application first registers to receive Raw Input from a particular device type, then when that device has data the application receives a WM_INPUT message in its event queue and the queue status flag QS_RAWINPUT is set. The application can then retrieve the data using the Raw Input API.

Raw Input was added in Windows XP, so is not available in Windows 2000
(Which is important -- see below.)

Raw Input and Qt

The problem with Qt (at least in 4.6.2) is that the version from the installer does not process Raw Input messages correctly. The way of collecting these the Raw Input messages is to set an event filter on the QApplication which gets the native windows messages. These can then be processed by the application using the Windows Raw Input API. However, the WM_INPUT messages are not being received by window/event filter the until some other event such as a mouse or timer event occurrs. Moving the 3D mouse causes no events to be received by the event filter but then if the normal mouse is moved (or key pressed etc)
all(?) the 3D mouse events arrive.

The problem can be traced to QEventDispatcherWin32?, which maps the windows messages onto Qt Events. Specifically, the line

   MsgWaitForMultipleObjectsEX(nCount, pHandles, INFINITE, QS_ALLINPUT,MWMO_ALERTABLE);

in the processEvents function of QEventDispatcherWin32. This is blocking and not waking on WM_INPUT events.

The windows documentation for this notes that QS_ALLINPUT, which includes QS_INPUT, “does not include QS_RAWINPUT in Windows2000.” Therefore, the problem is that the binary Qt installer is built to work on all windows platforms, so during the compile _WIN32_WINNT is defined to something less
that WinXP? (0×501) so that raw-input messages are not handled.

The fix is to re-compile Qt passing -D _WIN32_WINNT=0×501 to configure so that the WM_INPUT messages are correctly passed to the event filter when the 3D mouse is moved.

So this is not really a Qt bug, but rather that Qt is compiled for an "earlier" Windows platform which does not support Raw Input.

Sample Code for Qt and the 3D Mouse

I updated the 3DConennexion code for MFC to work with Qt. It uses an event filter to pick up the raw windows events then extracts the 3D Mouse information. This is then available as a Qt signal.

The zip file below contains a very simple application which displays the motion data in a window. This was tested with Qt 4.7.1.

3DMouse.zip

A script for adding test cases to a CppUnit test class

One of the problems with using CppUnit as the unit test framwork in C++ is that adding a single test requires updating the code in two files and in total 3 places. A declaration in the header file, a macro in the header file to regsiter the test and the test itself. This makes adding a test too hard, which means the tests themselves become too big because it is easier to add extra asserts to an existing test than to add a new test. Some of the newer C++ frameworks overcome this, but we have been using CppUnit for a long time and have a lot of tests already written so I do not really want to change frameworks.

To fix this I created a small Python script to add a new test, so working from the command line, I just need to write

newtest TestCase testMyNewTest

and the declaration and prototype are created. I don't need to even look at the header file I can just go straight to the definition of the test function and add the test.

The header file for a CppUnit test case looks like this:

class TestCase : public CppUnit::TestFixture

{

CPPUNIT_TEST_SUITE( TestCase );

    CPPUNIT_TEST( testSomething );

    CPPUNIT_TEST( testNextBit );

  CPPUNIT_TEST_SUITE_END();

public:

    void    setUp();
    void    tearDown();

    void    testSomething();

    void    testNextBit();

};

And the source file

CPPUNIT_TEST_SUITE_REGISTRATION( TestCase );

void TestCase::testSomething()

{

 // test code here

}

void TestCase::testNextBit()

{

 // test code here

}

The script is very simple, it searches for the last CPPUNIT_TEST macro in the header and adds a macro for the new test case. Searchs for the last test function, and adds a test after that. Then it adds a defintion at the end of the source file.

The script makes two assumptions

• The name of the class matches the name of the file. So the above test class would be in TestCase.h and TestCase.cpp
• The test class already has at least one test function when the script is run. Since I use a separate script to generate the boilerplate test class with a single test this is always true in my case.

The script is available here in case it is of use to anyone else: newtest.py

A CruiseControl monitor for QtCreator

We use CruiseControl to run automatic builds after every commit and I thought it would be nice to monitor the build status from within QtCreator. I also wanted to try creating a simple QtCreator plugin to see how easy it would be to extend it. I already use the Firefox add-on to monitor CruiseControl which does all that I need for a monitor so I just wanted to port that to work in QtCreator.  The plugin adds a red/green icon in the left hand bar to indicate the build status and a tooltip to show the status of individual builds.

The tutorial from VCreateLogic was very helpful setting up a basic plugin.  This tutorial describes how to set up a project, how to add menu items or new settings. So the basic configuration was straightforward.

The Welcome plugin which is part of QtCreator has a class for getting rss feeds. This formed the base for getting the data from the CruiseControl server. The main change required was an update to how the xml is parsed. Also, my first version worked fine when getting data from a test file but would not get data from the server. The original does not set the port, since I assume all normal rss feeds use the default port 80, but our CruiseControl server does not use port 80.  Once discovered this was trivial to fix.

The hardest part was finding out how to add the display widgets to the left hand bar in QtCreator.  A bit of poking around found a modemanager which manages the widgets. That seems to work but I am worried about API changes in future versions since there seems to be no official documentation for the API.

The source for the plugin is available on http://code.google.com/p/qtcreator-cc-plugin/

At the moment only the monitoring function is implemented.  Hopefully I will be able to find time to add functions to restart or stop builds from a context menu.

Qt Creator: A multiplatform IDE for creating Qt applications

QtCreator is a powerful integrated development environment from Nokia for creating applications with C++ and Nokia’s Qt Framework. It is available on all the desktop platforms supported by Qt (Windows, Linux and Mac). QtCreator can be used with either gcc or the Microsoft C++ compiler on Windows and the integrated debugger uses gdb or cdb depending on the compiler.

When QtCreator was released there were quite a few comments asking why another IDE was needed; why not just use VisualStudio, it is powerful mature and the `industry standard.’ Having tried to use Visual Studio in the past and now having used QtCreator for a couple of months the answer is clear.  If you are doing development with Qt then QtCreator is a much better environment.  This is principally because it is tailored specifically to Qt development so supports Qt directly and is not encumbered by masses of features which are not needed for C++ development.

•    Supports Qt project files so you don’t have to struggle with two different project management systems.
•    Has Qt Designer embedded directly into the IDE.
•    Good support for source control systems.
•    Built in help for Qt.
•    Simple project settings
•    Multi-platform.

The multiplatform nature of Qt is a big advantage for me. I now have identical development environments on both Windows and Linux, so switching between them is easy. If there is a bug which only shows up on the Windows version I can easily debug it on Windows, then switch back to Linux to run valgrind.

One of the features I really like in QtCreator is the locator bar. Opening files in an IDE has always been an issue for me. The file open dialog is so slow, and having to search through a large tree of files also takes forever. I like to be able to just type the name of the file, preferably with some form of auto-completion. With QtCreator this is possible. Ctrl-K takes you to the multi-function locator bar then just typing the name of the file will show a list of matches, hit enter and the file opens in the editor. It takes just a couple of seconds to open any file in the project, without having to take your hands off the keyboard.

The code navigation and search function is also very well done. There are function keys for header-source switching and for following symbols to definitions. One of the more useful functions is “Find usages” which searches the code model for uses of a particular function. The searches based on the code model are almost instantaneous even with a large project. There is also a multi-file full text search which is a bit slower.
The code editor has all the features you would expect, syntax highlighting, code folding and auto-completion. It is quite similar to Kate, which I used for a long time.  The only thing I miss is the block selection mode.

However, QtCreator is not as mature as some of the other IDEs, being much newer and does have some inconveniences. Starting the debugger with a large program seems slow, much slower than VisualStudio for the same executable. So far this has not been a big problem for me as I don’t use the debugger a great deal.
When a large project is loaded QtCreator seems to use lots of memory. If this gets swapped out due to some other operation on the machine then switching back to QtCreator can take several seconds with the interface stops responding for some time. This happens on both Windows and Linux.

There are also a few more minor annoyances
•    The diff viewer with the source control integration is rather basic
•    There is no support for “code snippets” – the source directory has a “dead” plugin and version 2.0 has some code-paster support based on shared files.

But it is likely that these will be improved in the future. Also, QtCreator supports plugins for extending the functionality. In fact most of the built-in functions are actually implemented via plugins. So far I have not found many 3rd party plugins but I would expect more to appear.

Development environments for C++

I have not been a big fan of IDEs. Several times I tried to get used to Visual Studio but always found it big, slow and not suited to the way I work. For a long time my development environment on Linux consisted of multiple instances of Kate (a KDE text editor) each on a different desktop and several small scripts. I would have source files in one editor and headers in another, so when working on a single class a simple keystroke to switch desktops would also switch header and source files. An “edit” script would open the file in the right text editor from the command line – I don’t think I ever used the file open dialog to open a file. Searching across multiple files was done using the GNU id-tools; very fast and easy to filter the output. As others have said, UNIX is the IDE.

Another requirement issupport for writing unit tests and Test Driven Development. With multiple editors open, I can have the test code in one editor and the source in another so switching is simple. A simple script “tests” run from the command line, builds the source, builds the tests and then runs the test cases.  Another script creates a new test function. We use CppUnit for testing, which is OK but does need a lot of boilerplate. Adding a new test requires not only the test function itself but also a declaration in the header file and a registration macro so the function is called by the test suite – this  is C++ so no reflection to automatically pickup the test functions. Since the boiler plate is always the same, except for the function name it is an ideal candidate for a script:

addtest  MyTestClass testFunctionOne testFunctionTwo testFunctionThree

would add three new functions and their declarations to the test class.

So basically, I was happy with my development environment. But there were two problems. I also have to develop on Windows and even with cygwin the environment there always seemed slower and I had to use a different editor. Secondly, all my colleagues use Windows with Visual Studio.  I see them struggling with basic tasks; they ask for my help on some coding problem so I suggest that they open a particular file and 2 minutes later they are still faffing around clicking through directories in a file open dialog. The majority of this problem is probably their knowledge of the environment rather than a problem with VS itself.  But, Visual Studio does have some significant problems for our C++ development.

We use Qt, which has its own project management system. The project is defined in .pro files which are then converted into makefiles by qmake. Visual Studio project files (vproj) files can also be made by qmake, so when a project is updated, the visual studio build files need to updated and then re-loaded. This means we don’t use Visual Studio’s project management functions. Even if we didn’t have this restriction I don’t like vproj files anyway because they are hard to manage in a version control system because they are difficult to merge. The second problem with Visual Studio is that it has no specific support for unit tests.

As a senior developer in the group I have some responsibility for the tools that the other developers use.  They want to use a nice graphical IDE – because it is easier. But I see all sorts of productivity problems in the way that they use it. We have a large code base, use Qt for the GUI and VisualStudio does not seem to handle this well at all.

So what I wanted was a development environment suited to large C++ Qt applications where I could use the same tools on Windows and Linux. It should have enough graphical features to not frighten the junior developers used to their cozy WIMP world but should have enough keyboard shortcuts to allow efficient development and be customizable for our specific requirements.

Then Trolltech released Qt-Creator.  While clearly not as comprehensive as Visual Studio, it is specifically targeted at the type of development that we are doing.  I have now switched to Qt-Creator and the next post will be on my experiences so far.

Gardening, not architecture

"Gardening, not architecture"

This Oblique Strategy caught my attention while flipping through the iPhone version of Brian Eno and Peter Schmidt's deck of cards for out-of-the-box thinking. Recently I had been thinking about Software Architecture, the role of an architect and how it relates to the actual work of producing working software. My official title is "IT Architect" but that is more a reflection of the ideas of the HR department than a description of what I actually do.

Our group is responsible for a fairly large engineering software product which has been through many release cycles. We have a large legacy code base in both Fortran and C++ which we need to maintain and extend, adding new features in response to customer requests. Large architectural changes are simply not practical or economically justifiable. The design and "architecture" evolves over time rather than being the result of some grand architectural vision. So what is the role of an architect in this environment? I think that the metaphor as it is commonly applied is flawed.

"Gardening , not architecture" seemed to be a more interesting description of the way software evolves. Often, while updating a module I come across code that seems to have become overgrown. Dead code which is no longer used has been left cluttering up the source files and needs pruning. Poorly formatted functions that need tidying up. Adding new features is a process of evolving the software in a new direction.


If there is a role for an Architect then the best description that I have seen is that given in "The Software Architect's Profession." They use a direct analogy with build architects and argue for separate professions of architect, software engineer and programmer. The architect would be the customer representative and "superintend the construction". I like this view because of its orientation towards the customer and away from the technological details. This description seems more like that of a Program Manager than the senior programmer which it usually seems to mean.