Thursday, February 14, 2013

QtSerialPort in Qt 5.1

Hi everyone,

I would like to introduce you to a project I have been working on lately under the Qt Project umbrella. Hope, you find it useful. :)

History

 

The serial port technology is quite old, but still in active use today. It is mostly used for communicating with modems, sending control commands and so forth where the universal serial bus is not necessary. I am sure many of you recall the time when soldering the pins and cable manually to get it right for the embedded boards. :-)

About a year ago, I was looking for a library for my project to handle the serial port communication with the modems I needed. As I had been about to make a touch friendly UI with QML, I thought it would have been nice to have a Qt solution. It would have been phantastic to communicate with the modem on my Windows, or Linux host.

There has been an old library around for a while, called QExtSerialPort. Unfortunately, its API looked weird and the project had been abandoned for a while without people contributing to it. Then, a chinese person began to fix certain issues, but still: the design had been carried for long from the old ages.


Serial Port Cable


Qt Project Playground


The Qt Project established a nice opportunity for contributors to start a new project, or move an existing to the Playground area. There was a discussion on the development mailing list to figure out the rules for handling this. Around the time, when I requested the QtAudio3D project into the Playground area of the Qt Project, there was a russian person, Denis, who initiated a community project into Playground in his own pastime to make a great developer experience for serial port based projects across platforms.

The library was supporting only Qt5 back then, and my project was based on Qt4. This is understandable as Qt5 was not realeased for the time, so we were unable to take the risks of the moving target. I decided to give the project a go and make support for Qt4. Not much later (i.e. 1-2 weeks), I got the first changes into the Qt Project, following the Gerrit workflow. By that time, it was more or less possible to use QtSerialPort with Qt4.


Gerrit - Code Review Tool

Our work has been achieved by using the Qt instance of the awesome Gerrit codereview tool. Almost every commit got at least one review, but many times two, or sometimes even more. I think that is nice, and helped to guarantee the quality of the work even without having a Continous Integration (CI) in place for the time.


As time time had flied, I had more and more feature requests and own changes for those to implement. That is how the library kept rolling, getting new features and bugfixes. The API was also a bit too low-level, and I began to send patchs to refactor the API to be more Qt style'ish. As I am a geek, I instantly wrote a command line based example to enumerate the devices. It was necessary for me to get the project tested on my pandaboard where the UI was not available. There were also important features added like support for gadget serial driver on Linux and so forth.



The command line enumerator example

The initial plan was to make QtSerialPort available as an add-on module for Qt 5.0. This had been proposed on the development mailing list last year. You can read the platforms and environment supported in that threa. Unfortunately, the Qt Project did not have enough capacity for this integration. Likely, it was also good for us because we were able to clean up the remaining uncertain bits of the API. I am not claiming it is now perfect, but it looks much better than it used to be. :-)

Future plans

 

As you can read in the topic of this post, QtSerialPort makes it for Qt 5.1. This has been approved by Lars, the Qt Project Chief Maintainer, in the aforementioned thread. This is a really great and exciting news for us. It is also a very good prove for the Qt Project that the idea actually works. Community people can contribute with new modules under the same umbrella with the companies behind the Qt Project.

The main short term plan is to get the integration right in terms of Jira and Gerrit to move out of the Playground scope. As the CI integration has happened lately, the project is now also covered for build tests, so we cannot end up in situations like having regressions as before. It is not so simple to maintain such a project like this without proper CI integration because we experienced many regressions on one platform when we tried to fix issues or add new features.

FTDI USB-Serial adapter


As you may already know, it is not simple to test the serial port projects automatically. There had been an initiation for a Qt Mock project from me last April, but it did not yet manage to reach any progress. Nevertheless, we are working on making tests available as much as possible.

My personal plan is to write a KDE frontend for the library, preferrably also a terminal emulator. We already have a terminal emulator example in place to represent the developer and user experience for the project. It is not full-fledged, but it has been working nice for the basic needs. You can set the baudrate, parity and stop bits, flow control et cetera. Something like that would be nice in the KDE project, or perhaps even a plasmoid. There has been a CuteCom around for a while, but when I briefly discussed the project with Alexander Neundorf last summer at aKademy in Tallin, it did not support all the platforms we need.


Terminal example


As you may already know, the QNX platform is also getting closer to a Tier 1 stage in the Qt Project. Blackberry, formerly known as Research In Motion, and KDAB made an excellent job in that area. I have been considering to add a QNX backend support for the project at some point.

Kudos

 

First of all, many thanks to the Qt Project!

Secondly, many thanks to all the contributors to QtSerialPort!

Authors

The reviewer statistic is generated by the following command: git log --grep '^ *Reviewed-by:' | sed -n 's/^ *Reviewed-by: *//p' |  sed 's/ *<.*>//' | sort | uniq -c -i | sort -n -r


Reviewers




Sunday, February 10, 2013

Geek'ish application development for Blackberry 10

Hi there again,


As indicated in my previous post, I am now about to summarize my developer experience for the platform Blackberry 10. As written before, the regular way of developing applications for this platform is to use the Momentics IDE or QtCreator.

Those environment usually generate the necessary qmake build system files, bar files for you to a certain extent I guess. Admittedly, I have never used them for this purpose, so this is just a gut feeling. Anyway, this is not so important for the scope of this post.


Goal



When I had started to develop my first application, I made some research if it had ben possible to develop applications the way I like: command line and cmake. For those of you, who are not familiar with cmake, it is a nice cross-platform Makefile generator thoroughly used in many projects. One open source reference is KDE for this.

Interestingly enough, I got some messages in private after my cmake based projects that people found those files and used in a similar fashion one by one. This is nice, but some explanation is necessary for newcomers. So, let us dig into more details about this.


CMake Toolchain file



In order to make cross-platform development with cmake, usually one has to produce a correct toolchain file. I looked around on the internet, and there was some examples available for QNX. I took one of those essentially to start off with. That is also one reason why it is a bit bloated, and a much simpler would be enough for achieving what we wish to.

If you are not interested in the underlying operation, you can skip the explanation below, and you can just use my toolchain file right away.

The important changes I made were the followings:


  • Use the proper toolchain binary

Use the 'qcc' build wrapper. It is essentially a wrapper around the compiler, linker, and so forth. You can find more complete documentation about it here.

This is a very important change because I started with the "ntoarmv7-g++" which is a link to the "arm-unknown-nto-qnx8.0.0eabi-g++-4.6.3" toolchain file. I also tried the latter directly, but I encountered crash for my applications even for a very simple test case without cmake. The program crashed even before entering the main function. It took me two nights at least to track the problem down.


  • Use the proper toolchain binary flags

Once I figured out I would have to use 'qcc', there were still some problems that I encountered, albeit slightly different. I had to realize the usage of certain flags is necessary. Once I made it work with the proper flags without any build system, I had to figure out the proper way of setting that for the toolchain file. Now, we have all the information necessary so here it goes the statement:

    -> SET(CMAKE_CXX_COMPILER_ARG1 "-Vgcc_ntoarmv7le -lang-c++)
  • Instruct cmake to use the toolchain file created

     -> -DCMAKE_TOOLCHAIN_FILE=/path/to/the/toolchain/file.cmake


  • The location of the toolchain file
 This is only my personal opinion, but I found it simpler to put the toolchain file into the project because then I did not have to carry that on to other machines I worked on. It is also simpler to ask other non-cmake developers to test my application if I pass a command to them. It is not a huge file, so it probably does not take up much space in the repository either. It does not clutter the project that much either in my opinion.


So this is the entire command I use from the build folder:

    ->  cmake -DCMAKE_TOOLCHAIN_FILE="../frontends/blackberry/cmake/Toolchain-QNX-8.0.0.cmake" .. && make VERBOSE =1


Find modules for Cascades libraries


This is also a very important step to find the Cascades libraries for your application. This was not a big deal, albeit I have not had time to solve this issue nicely either. So, there are rooms for improvements. :)


I have created separate cmake find modules for each library for the time being, but perhaps it should be handled more like Qt4 with a joint find module, and one can request anything out of that. So, there would be a macro with a few parameters. It would not be then necessary to copy and paste the code. If you are not up to that contribution, you probably better go with the files I created. You can find them below.






Deploying the application to the device



I wrote a primitive python script for this after gathering all the information what flags to use. Here you can the result. This one is more or less just a hint as it cannot be used that directly. I chose python because then I can also use this on Windows and elsewhere as python is a nice cross-platform scripting language.

Essentially, there are several tools that have to be used for installing and then launching the application on the device from your host system:

  • Set up the Blackberry NDK environment

I personally made an alias for this, but here you can find the comment:

        -> source /opt/bbndk/bbndk-env.sh

  • blackberry-connect
I have not tried to develop over Wi-Fi because I use mobile internet, so that was not an option. I also prefer the usb link as it is faster and more reliable. I made aliases for these, too. You have to supply a proper public ssh file. I think the default length is not proper, but a new one can be generated easily. Here you can find the command for setting this all up:

        -> ifconfig usb0 169.254.0.2

        -> blackberry-connect 169.254.0.1 -password devicepassword -sshPublicKey ~/.ssh/id_rsa_rim.pub

  • blackberry-nativepackager 

Here you can find an example how to use it: 

        -> blackberry-nativepackager -package -target bar test.bar /path/to/the/bar/descriptor/xml/file.xml -devMode

  • blackberry-deploy
Here you can find an example how to use it:

        -> blackberry-deploy -installApp -device 169.254.0.1 -launchApp -password devicepassword -package test.bar



Debugging from command line



A simple ssh connection can be used for logging into the device once the development mode is switched on in the settings. It has to be done after each boot as of now on my devalpha, at least. Make sure the connection is established properly and the blackberry-connect process is still up. Here you can find the command for that:

    -> ssh -i ~/.ssh/id_rsa_rim devuser@169.254.0.1

Once this is done, there are some tricks to get information about QML syntax issues, C++ crashes, reading your intended log file, and so forth:

    -> slog2info -w 

    or/and


    -> cd /accounts/1000/appdata/${id_from_the_bar_descriptor_file}/logs && cat log

You can also get the screenshots created on the device by pressing the two volume buttons simultaneously:


    -> scp -i ~/.ssh/id_rsa_rim devuser@169.254.0.1:/accounts/1000/shared/camera/IMG_*.png /path/to/the/directory/to/store/the/screenhots



 

Application signing for AppWorld


Here you can find the command I used:


    -> blackberry-nativepackager -package -target bar test.bar /path/to/the/bar/descriptor/file.xml -sign -cskpass devicepassword -buildId 1


Note: Do not forget that you have to increase the -buildId value for each new signing.

 

 

Testing someone else's application


This is also a common request that someone would like to get feedback about the application before even getting approved in the AppWorld. In such cases, you may request your friends and acquaintances to test it. This could also happen vice versa so that you are asked for testing an application and provide feedback. Hence, it is useful to know how you can do that. Here is the command I used.


        -> blackberry-deploy -installApp -device 169.254.0.1 -launchApp -password devicepassword -package test.bar



Conclusion


Like many things in the life, this is also possible. I do not find any issues with this workflow for application development, but I am a bit strange developer. I am glad to have found the way I like spending my time. Many thanks go to Bill Hoffman from Kitware for patiently helping me with cmake issues on the cmake mailing list.


Hope, it helps some people out there.


Happy Blackberry 10 hacking! :-)

New applications for Blackberry 10

Hi everyone,


Introduction

I have been planning this post for a while, but I had some real life troubles and also other duties. Finally, I have some time to write this down briefly!

In short, I have created two new applications for Blackberry 10. I tested them on my DevAlpha device. The first one became the candidate for getting a Limited Edition device in the end.


Wiki Reader

As I have been a wikipedia fan for a while, I thought this would be a good idea to help myself and others to have an application for reading wikipedia pages easily. There was an application on Harmattan (N9) called "Cutewiki" available in Ovi Store. I had been using that a lot because I think it is a great application. As the author did not plan to port that to Blackberry 10, I wrote one for myself.

The source code is available under the KDE umbrella. The application was approved in AppWorld, but here you can find some screenshots embedded as well:




Searching for articles

Reading articles


Mrdanga Player

I had been playing bass guitar for many years during my student years, and I have been a fan of mrdanga recently. There was an application for Harmattan (N9) called "Finger Drums". I felt in love with that application after the first checkout. It is an awesome piece of software with lots of fun!

This drove me to write an application for Blackberry 10 to enable people to get to know what mrdanga is and how it sounds. It is a lot simpler emulation than a whole drum kit. I think it is still nice for those who like this instrument, and potential newcomers that can get to know it this way. There is also a nice video on youtube if you would like to check out what the masters can do with this instrument. :)

The source code is available under the KDE umbrella. The migration is now ongoing to Playground/Mobile where the wiki-reader also resides.

The application was submitted to AppWorld, and it is in review stage. For those who would like to try out the latest version, I uploaded the package to my dropbox. I would also like to share some screenshots here.

Splash screen


Mrdanga heads with darker background

Mrdanga heads with brighter background



Kudos

Many thanks go to RIM for providing DevAlpha and Limited Edition devices to the community!

For sure, many thanks go to the KDE Project as well for providing the source code repository, bugtracker, and the whole KDE infrastructure for this!

There will be a follow up post soon about developing Blackberry 10 applications from command line with cmake. Stay tuned!