Using Tarsnap GUI on OS X

Tarsnap logo

For the past year I’ve been working on Tarsnap GUI, an open source cross-platform frontend for the Tarsnap backup service. Tarsnap is an online, fully encrypted, deduplicated, online backup solution, that’s been around for close to 10 years now (unlike other online backup offerings with a lifespan smaller than 5 years). Starting with day one, full source code for the client tools was available for review and to establish trust. A bug bounty program is available and any kind of reporting is taken seriously, promptly responded and adequately rewarded by the creator and principal developer of Tarsnap, Colin Percival. Tarsnap uses a prepaid model, you add credit to your account and you’ll only pay for what you use and nothing more, no annual or monthly subscriptions. For the moment, the price per MB (SI notation) of storage (used monthly) and MB transferred is 250 picodollars each. The backend infrastructure is powered by AWS.

Tarsnap was originally offered as a command-line suite of tools, in true allegiance to the Unix philosophy, very similar to the tar utility, for maximum flexibility and control over your backup routines and of great relevance for the server world. Die-hard Unix users and admins see no distinction between server administration and personal workstation and thus nothing is stopping you from backing up your personal machines and workstations the same way you would a networked server. Tarsnap GUI comes to the rescue here in filling whatever gap might be between the server and desktop workflows and for the people that like to keep real work inside the Terminal and all else contained in easy to use, slim, yet robust GUIs. What follows is a small tutorial on how to get started with Tarsnap GUI to back up your OS X desktop. Setting the platform differences in the Installation and Scheduling sections aside, this tutorial is very relevant to the other supported platforms too.

Register with Tarsnap

First and foremost you need to register on the Tarsnap website and add some credit via the preferred method of payment (Credit card, Paypal and Bitcoin are accepted methods). Five bucks would suffice for a start. Read a bit on the homepage and getting started page just to get acquainted with Tarsnap. Tarsnap doesn’t ask, thus doesn’t need to store any personal information besides your e-mail address.

Installation

At the moment of writing this tutorial, Tarsnap GUI version 0.8 and Tarsnap CLI version 1.0.36.1 are the latest offerings and installing from source is the preferred method of installation for all platforms, although we are working towards changing that at some point.

Installing from source on OS X is pretty easy thanks to Homebrew, an alternative to Mac ports. Once you installed Homebrew or updated to the latest version (tarsnap-gui was recently added to the homebrew-gui repo), installing Tarsnap is as simple as executing the following command in Terminal:

brew tap homebrew/gui && brew install tarsnap-gui && brew linkapps tarsnap-gui

What this will do is fetch the latest stable versions of the Tarsnap command-line utilities and the GUI front-end along with their dependencies from Github, compile from source and then add useful symlinks to the appropriate system directories. The Tarsnap command-line utilities are symlinked in /usr/local/bin and a symlink for Tarsnap.app should be in /Applications.

If Homebrew is not a desirable option for you or you’re reading this with *Linux or *BSD in mind, the steps for installing manually from source are best described in the INSTALL dist file. Some BSD flavors like FreeBSD, PCBSD have already incorporated tarsnap-gui into their repos or ports trees, or plan to like OpenBSD in next version, so it might be worth to look for it there before deciding to proceed with the manual steps. Upstream Linux RPM and DEB packages are being considered for a change.

Wizard setup

Upon opening /Applications/Tarsnap.app you will be greeted with the following:

Tarsnap-tutorial-1

This is the setup wizard and will help you get started with using Tarsnap in no time. The first step for the wizard is to find the Tarsnap command-line utilities that are used to talk with the Tarsnap servers. In the Homebrew case, they will be located in /usr/local/bin and depending on the PATH environment variable for your current user the wizard will either direct you to the Advanced page to set that directory manually or proceed right next to the next step for the case where you already have it in your PATH. You don’t have to worry about the other settings in the Advanced page for now.

Tarsnap-tutorial-2

Tarsnap-tutorial-3

 

On the next screen you will be asked if you used Tarsnap on this machine already, assuming this is a fresh setup you should click on No. If you already used Tarsnap in the past and have a key for that machine (it is good practice to use a different key for every machine), I’m guessing you already know what you’re doing and thus you can proceed the other way around. Upon clicking on No you end up to this screen:

Tarsnap-tutorial-4

 

This is where you fill in your Tarsnap account credentials from the Registration step. When you click on Register machine, your encryption key is created locally and an accounting record using the Machine name specified is created on the Tarsnap server and associated with a signature of the key. This is used for generating your service and credit usage reports per machine (or key signature, the machine name doesn’t matter much and can be anything you want), for your convenience, on Tarsnap.com. The actual key used for encrypting your data never leaves your machine. Your backups are safe as long as you will be able to hold on to this key privately. If you lose this key your backup data is gone forever. The actual path where the key is stored on your system, along with some useful advice, will be revealed to you in the last screen upon successful registration.

That’s all it takes to quickly set up Tarsnap on your desktop, rather simple right? Let’s proceed by taking a first look at the main app window.

First impressions

Tarsnap-tutorial-5

The first thing you’ll see is the empty Backup pane. The Backup tab is used to quickly put to rest arbitrary files and directories of your choosing to Tarsnap. Think of it like a convenient one-time drop bin, where you can easily drag and drop important files, set a name for the archive, hit Backup button and forget about it.

Tarsnap-tutorial-6

Clicking the Backup button will result in an archive named Tarsnap-tutorial consisting of the files and directories added to the list. In a matter of seconds I have an off-site, encrypted backup of the resources in this tutorial, awesome right? Now let’s look for the archive just created, leading the way to the next pane, Archives:

Tarsnap-tutorial-7

This pane lists all the archives created for the current machine/key, providing for means to manage, inspect and restore archives. You can already notice the Tarsnap-tutorial archive. Let’s double click that item to get a detailed look:

Tarsnap-tutorial-8

You can notice several things from this view that describe an archive in Tarsnap terms:

  1. The archive name, Tarsnap-tutorial in this case. The name can be any arbitrary combination of characters, excluding the null character and must be unique amongst all the existing archives.
  2. The archive size, 4.83MB in this case. This is the original size of all the items included in a backup, before deduplication  and compression.
  3. Unique data, 2.15MB in this case, what you pay for. This is the archive size post deduplication and compression. As you can see, Tarsnap is smart enough not to waste your credit on duplicate data, even for single archives.
  4. A command. This is the exact command-line used by the application to create the archive. Every single action executed by the GUI app has a direct command-line equivalent, which can be reviewed in the Help -> Console pane. Total transparency, control and debugging capability. You can even use this to learn how to operate Tarsnap at the command-line.
  5. Contents listing. This list contains all the source files and directories included in an archive.

Every Tarsnap operation is accompanied by a status bar message. When Tarsnap is working with the server a loading circular orb will animate to indicate activity. All of the status messages are logged and can be reviewed by expanding the Journal with a click on the arrow head in the status bar.

Tarsnap-tutorial-9

Using Jobs

The obvious evolution from simple on-demand one-shot backups are Jobs, logical definitions for important locations in your file system that you know are going to be backed up regularly. These could be your Documents, Work or Desktop directories. Let’s define one now for the Tarsnap tutorial directory used previously, by switching to the Jobs tab, clicking on Add Job button and filling up the Job specifics:

 

Tarsnap-tutorial-10

So a Job is:

  1. A name, Tarsnap Tutorial, in this case. This name is prepended to the archive names for this Job.
  2. Files and directories selected in the File System tab. This is what you want to back up as part of this Job.
  3. Options particular to this Job. We’ll take a look at this later on.
  4. Archives pertaining to this Job. None yet thus pane is disabled.

Let’s change fact no. 4 and create the first backup for this Tarsnap Tutorial by Saving the Job and then clicking the Backup button for the Job entry that appeared in the list:

Tarsnap-tutorial-11

A couple seconds later:

Tarsnap-tutorial-12

Double click the resulting archive in the updated Job -> Archives pane and let’s take a look at the result:

Tarsnap-tutorial-13

A Job archive has all the properties as the previously created on-demand archive. A couple observations are worth noting:

  1. The Job_ prefix is prepended to the archive name;
  2. The archive has a reference to the Tarsnap Tutorial Job. You can jump to the Tarsnap Tutorial job by clicking on the Job icon in the archive listing or the Job: label;
  3. I’ve added 5 extra screenshots to the directory since the previous on-demand archive. If you take a look at the Unique data label you’ll notice that only 1.25MB of data has actually been uploaded to the Tarsnap server and thus you’ll only be charged for that. This is the deduplication magic in action that works between all archives created with the same key and will save you a lot in the long term when backing up from the same sources of data in what amounts to, basically, incremental backups.
  4. Tarsnap also compresses your data after deduplication step for maximum storage, bandwidth and resulting cost efficiency. In this case I’m backing up PNG files which are already compressed data formats (also MPG, AVI, MP3, MP4; in general any modern media format has some level of compression), but if I were to backup source code, text files, documents and other raw formats the amount of savings will be even greater. This also means that it’s pointless and even detrimental to the efficiency of Tarsnap if you’re compressing your files manually before using Tarsnap.

This is the basics of using Jobs with Tarsnap GUI. What you should do next is define individual Jobs for each of your important file system locations like Documents, Pictures, Desktop and your Work directory. You can back up all your Jobs at once by clicking on the Add job button drop down and Backup all jobs or selecting all Jobs and hitting CMD+b keyboard combo. For a full listing of keyboard shortcuts available see Help pane.

Next we’re going to take a look at scheduling automated backups for Jobs.

Scheduling

At the moment, scheduling automated backups for your Jobs requires a bit of knowledge about your Operating System scheduler and some degree of manual work. Given that Tarsnap GUI is cross-platform (OS X, BSDs, Linuxes and Windows in the future) and that every platform usually differs in the best choice for scheduling, it’s quite tricky to code a solution that applies elegantly and reliably for all, so for the meantime we’re left with a manual approach. I’m working towards changing that in a future release, if you want to find out when that happens it’s best you either follow the project on Github or subscribe to the tarsnap-users mail list (links at the bottom of this post).

The best scheduling method on OS X is Launchd. Tarsnap GUI executable has a command line parameter called –jobs:

/Applications/Tarsnap.app/Contents/MacOS/Tarsnap -h
Usage: Tarsnap [options]
Tarsnap GUI - Online backups for the truly lazy
 
Options:
...
-j, --jobs Executes all jobs sequentially that have the 'Include in scheduled backups' option checked. The application runs headless and useful information is printed to standard out and error.

You can probably figure out what’s coming next. We’re going to schedule Tarsnap –jobs with Launchd to run on a schedule. Before we do that we need to enable Tarsnap Tutorial job for inclusion into automatic backups (-j). There’s an option for that, named Include in scheduled backups:

Tarsnap-tutorial-14

The option is disabled by default and you need to check it as shown in the screenshot.

For the last step you need to open Terminal and download a sample Plist file that describers the invocation routine for Launchd to the appropriate location and enabe it using launchctl command. You can take a look at the sample plist from the repo on Github:

curl https://raw.githubusercontent.com/Tarsnap/tarsnap-gui/master/util/com.tarsnap.backup.plist > ~/Library/LaunchAgents/com.tarsnap.backup.plist
launchctl load ~/Library/LaunchAgents/com.tarsnap.backup.plist

This launchd script will invoke Tarsnap -j every Sunday at 10AM or the next wake from sleep or system boot right after that (for cron fans, beware that crond doesn’t do that on OS X). So the next time when that date comes around all Jobs that have the Include in scheduled backups option enabled will be backed up. Let’s do a test run now and see what happens:

/Applications/Tarsnap.app/Contents/MacOS/Tarsnap -j

Assuming you have notifications enabled on OS X (otherwise you’re stuck with the output in Terminal) this is what you’ll see:

Tarsnap-tutorial-15

When executing Jobs in the background, Tarsnap GUI will have an icon in the menu bar and desktop notifications will notify you of what’s happening. If you click the tray icon or the  notifications the app will fire up.

Now if you’re reading this with Linux or BSD in mind, all you have to do to schedule your backups is replace Launchd with something like crond.

Restore

What use are backups though without a simple method for restoring backed up data in case of need? Tarsnap GUI takes care of that without much fuss. Let’s delete the Tarsnap tutorial directory and attempt a restore from the latest Tarsnap Tutorial backup by clicking on Restore latest archive button for the Job.

Tarsnap-tutorial-16

Clicking Restore immediately commences the restoration of files in the last Tarsnap Tutorial backup.

Tarsnap-tutorial-17

A couple seconds later, lo and behold, I have my work on this tutorial back. Not much of a surprise huh? Nobody likes surprises when it comes to backups anyway.

Afterword

While Tarsnap CLI and Tarsnap GUI is capable of much more than what is laid out in this primer, I hope this was enough of an introduction to get your feet wet and encourage you to start exploring the other things you can do with this setup on your own. If you prefer reading this tutorial in a PDF version here you go.

Relevant links:

Tarsnap website: https://www.tarsnap.com

Tarsnap on Github: https://github.com/Tarsnap

Tarsnap mail lists: https://www.tarsnap.com/lists.html

Tarsnap GUI releases: https://github.com/Tarsnap/tarsnap-gui/releases

Tarsnap GUI Wiki: https://github.com/Tarsnap/tarsnap-gui/wiki/Tarsnap

Tagged with:
 

Tarsnap GUI v0.8

Tarsnap logo
New year brings fresh meat. I’ve just tagged version 0.8 at Github:

https://github.com/Tarsnap/tarsnap-gui/releases/tag/v0.8

This release continues on the track already paved by the previous release in terms of performance, code robustness and adherence to modern C++ practices and UI consistency, adds a bunch of new features like simulation mode, skip files flagged nodump in the file system and a persistent Journal, as well as numerous other improvements, fixes and adjustments throughout the whole spectrum. This release makes Tarsnap GUI leaner, faster and more robust overall.

More cool things have been packed packed into this release, so I urge you to read the announcement e-mail for all the juicy bits:

http://mail.tarsnap.com/tarsnap-users/msg01215.html

Take Tarsnap GUI v0.8 for a ride and let me know what you make of it:

https://github.com/Tarsnap/tarsnap-gui

Tagged with:
 

Tarsnap GUI v0.7

Tarsnap v07 OSX

I’ve just tagged version 0.7 at Github:

https://github.com/Tarsnap/tarsnap-gui/releases/tag/v0.7

This release improves on the earlier v0.6 release, adds a bunch of new features like desktop notifications, skip files and the ability grab the Tarsnap credit from the website, along with a slew of general improvements, fixes and UI refinements. Better looking, leaner and more robust.

Desktop notifications are especially useful for the scheduled job backups, so that you are notified when the backups have started, completed or failed. OS X displays the notifications like this in the Notifications Centre:

Tarsnap OSX notifications

These are arguably valuable for quickly reviewing scheduled backups that have completed while you were away.

I’ve also carefully refined the UI for high res/dpi displays so that it looks gorgeous. This is how casually browsing your archives in Tarsnap looks like on a 40″ 4k display:

Tarsnap OSX 4k

More cool things have been packed packed into this release, so I urge you to read the announcement e-mail for all the insight:

http://mail.tarsnap.com/tarsnap-users/msg01175.html

Take Tarsnap GUI v0.7 for a ride and let me know what you make of it:

https://github.com/Tarsnap/tarsnap-gui

Tagged with:
 

Johnny 2.0 (reloaded)

johnny

Johnny, the GUI interface for the popular John the Ripper password cracker has received quite some love this past summer in an orchestrated effort to pick it up and drag it beyond the stale 1.0 branch.

Johnny who

Johnny is the cross-platform Open Source GUI frontend for the infamous password security testing suite John the Ripper. It was originally proposed and designed by your’s truly in 2011 as a POC, then version 1.0 basic implementation was achieved by Aleksey Cherepanov as part of GSoC 2012. Nothing much else happened beyond the 1.1 fix release.

Johnny’s original aim is to automate and simplify the password testing/cracking routine across all major desktops with the help of the tremendously versatile and robust John the Ripper suite, as well as add extra functionality on top of it, specific to the desktop and GUI paradigms in contrast to the command line, like improved hash and password handling, multiple attacks and session management, easily define and test complex attack rules, visual feedback and statistics, all of it by building on the immense capabilities and features already offered by both JtR core/proper as well as jumbo flavors.

Johnny 2.0 reloaded

Fast forward to 2015, I finally got some spare time to turn my attention towards Johnny again in order to further the stated goal for Johnny in the previous paragraph. So I devised a fresh plan for developing Johnny further and reconsolidate the original mission. The development plan has turned into reality with the acceptance of Mathieu Laprise as a student coder for Openwall (the org behind JtR and many other cool projects) as part of this year’s GSoC iteration. The tasks in the roadmap were split between me and Mathieu and with help from my co-mentor Aleksey Cherepanov we proceeded to the actual work involved in rebooting Johnny.

Now that the summer has concluded, it’s time to draw a summary of the achievements:

  • Cross platform issues fixed across all latest versions of supported Operating Systems and desktops
  • The UI has been significantly revamped for improved usability, robustness and consistency and looks across latest desktop paradigms
  • Full translation and I18N support added (only French for now, contribute translations to your own language on github)
  • Attack session history and persistence, easier to define new attacks
  • Greater coverage of JtR core and jumbo functionality (fork, jumbo attack modes, hash format detection)
  • Improved input and output options (2john format conversion support, export to CSV)
  • Smarter Passwords table (ability to show hash format, filter, sort, include/exclude from attack)
  • You can now test passwords manually via the Guess button

Overall Johnny is faster, more robust, better looking and much more equipped and forward looking (code and internals wise) than the previous incarnation and resulted in a significant code/ui refactoring of the original codebase (maybe 80% rewrite). All of the goodness described above and more was delivered to users in three releases starting with a major version bump to 2.0 to reinstate the fresh reboot and outlook for the project. The latest release is v2.2 and is considered to be stable and feature packed enough to be called the official GUI for John the Ripper. There are binary packages for Windows and OS X and detailed source build instructions for the other platforms on the wiki page for Johnny, thus I urge you to give it a spin and leave feedback here, on Github (where the project is hosted and tracked) or on the john-users mailing list. As always, contribution of any kind is very appreciated.

Acknowledgementsgsoc2015-300x270

 

Thanks to Mathieu Laprise for his important and dedicated contribution to Johnny as a student coder for GSoC 2015 and we hope to hear back from him from time to time. Big thanks to the entire john-dev community and Aleksey Cherepanov. Also an extended appreciation goes to Google for their continued dedication to support Open Source and contribute big bucks in the process.

 

 

Johnny on Ubuntu

Johnny on Ubuntu

Johnny on Gnome 3

Johnny on Gnome 3

Johnny on OS X Yosemite

Johnny on OS X Yosemite

 

http://www.openwall.info/wiki/john/johnny

https://github.com/shinnok/johnny

Tagged with:
 

Tarsnap frontend released

 

For the better part of the last half year I’ve been working on a Graphical User Interface for the popular Tarsnap online backup service. The frontend enables desktop users to benefit from the same advantages that the Tarsnap service offers, admittedly only at the command line interface. These benefits include:

  • cost effectiveness;
  • flexibility and control of your backup process;
  • total openness on the client side;

While the Tarsnap service is of immense efficiency and conveniency for server and scripted backups thanks to its Unix utility design, that same characteristic makes it a good choice for wrapping a desktop application with a graphical interface on top so that more users can benefit from truly secure and well designed backups as well as to easily define and use more complex backup routines from a comfortable and lean interface.

Tarsnap-v0.5-OSX-Yosemite-Wizard

Tarsnap Setup Wizard

 

The application is cross platform, written in C++ with the help of the Qt SDK and is released under the BSD 2 clause license. It has been tested on OS X Yosemite, FreeBSD and most Linux distributions with a wide audience.

In the present state, the application makes it easy to:

  1. Configure Tarsnap on your system via a Setup Wizard;
  2. Make single shot backups from the Backup tab, for quickly putting some important files and directories at rest on Tarsnap;
  3. List, inspect and manage your backup archives from the Archives tab  ;
  4. Define frequent backups you attend to as a job from the Jobs tab; A job has custom backup paths and settings;
  5. Provides useful customizations and help via the Settings and Help tabs;

The current version is 0.5. There’s so much more to be implemented and ground to cover in both usability as well as Tarsnap options breadth and depth coverage. So stay tuned for further releases and announcements to this blog, @shinn0K, @tarsnap or the tarsnap-users mail list.

Project page and source code is available at GitHub: https://github.com/Tarsnap/tarsnap-gui

Head over to this overview Wiki page for extra details regarding the current release. Also read the announcement e-mail on the tarsnap-users mail list.

As explained in the wiki page and the announcement e-mail, there are no binary redistributable packages for now, mainly because there aren’t any for the Tarsnap command line client either. This might change in the future though.

As with any Open Source project, I encourage you to participate via discussion, feedback, code contributions, bug reports, fixes and platform testing. Basically any type of constructive and informative contribution is very much welcomed.

Tagged with:
 

Inspired by my previous post on using Qt Creator on non-Qt SDK projects, I went a step further and integrated the Linux Man Pages section 2(system calls) and section 3(library functions) as a context help module in Qt Creator and Qt Assistant. Having documentation on all of the Linux system calls and system libraries at one click away(F1) is a great advantage and boost to productivity when doing system programming on Linux(and not only), especially now that Linux has around 400 system calls and GlibC 1400 functions.

 

The Man pages have been bundled as a Qt Compiled Help module that is easily plugged into Qt Creator or Qt Assistant. The HTML pages used are the official ones available at kernel.org, the online version. I have removed the Google search and StatCounter Code since they are not necessary for this purpose.

The manpages are accessible by context help, when pressing F1 on a declared function like select or printf or by visiting the Help screen and browsing them alphabetically or by searching the index. You can find more information about them right there in the Help screen in the Linux Man pages category.

You can download a gzipped version Qt Compiled Help Linux Manpages from here:

http://trunk.shinnok.com/qt/manpages.qch.gz

Using it is pretty straight straight forward, open up Qt Creator and go to Tools -> Options -> Help screen. Switch over to the Documentation tab and then click on Add, like so:

Browse to your gunzipped manpages.qch location and click Open. You should now see an entry named org.kernel.manpages.1.0 that points to your supplied location:

Remember that if you remove or move the .qch to another location Qt Creator will not be able to find it anymore, since it doesn’t create a copy for itself. It’s best if you put in a place that makes sense in this manner.

If i see demand for this module, I’ll move the instructions on how to import the Linux man pages Qt Compiled Help module, as well as download links, to the main site(shinnok.com) and I will try and update them every so often once I hear of important changes to the man pages. Till then, this will be a blog simple post.

Happy hacking!

Tagged with:
 

 

Qt Creator

Leading rant:

Inspired by this post(read it because I am going to reference it) on the Wikitech mailing list I was inspired to write about Qt Creator, the IDE that I use for about an year now for almost any kind of C/C++ project and which I think it is pretty close to being a type 1 Integrated Development Environment, as written by Paul in that e-mail, which is a real and useful IDE, at least for my use cases. Qt Creator is a powerful and flexible IDE most useful for C/C++ code and especially when used in combination with the Qt SDK. However, due it’s extreme flexibility, Qt Creator can be tweaked to work with almost any kind of C/C++ source project, from the default qmake project files to CMake, GNU Autotools(the GNU Build System), standalone Makefiles or even custom commands for direct access command line steps for direct compiler and linkage directives.

While I prefer type 2 IDE’s for most of the day, which are powerful text editors like Vim for casual hacking or minor code modifications, which VIM excels at, especially with its modal mode, delving into an entirely unknown code base(especially a big one) or using new libraries and apis or for long coding sessions, in a simple text editor, even as powerfull as Vim or Emacs, things start to get inconvenient pretty easily and you will notice a slow down, mainly because of the poor code browsing, syntax highlighting(C++) and code completion capabilities that they posses. They are not to be blamed though and there’s also nothing wrong with them, they are simply text editors and not IDE’s. Since Vim is my favorite text editor, I tried various combinations of Ctags and CScope and all kinds of crazy hacks on top of that, just to be able to have code nagivation and code completion at least to some extent. You can find my Vimrc file in here, without Ctags and CScope, since that would make for entire post by its own.

However, I got tired of it after some time, especially because it doesn’t work that nice with some of the crazy quirks that C++ for e.g. exhibits and because I have to keep the databases that those tools use up to date frequently and manually, for every project that I am working on. For terminal only access, coding like this is fine, to some extent, however, this seemed really overkill on my desktop, especially since I am running an X desktop with a GUI and knowing that I could do better to be more productive in 2011.

Ok, let’s take a first look at how Qt Creator looks like in action, since I know I made you curious by now:

Now I’m going to list some of the strongest points and features that Qt Creator has and that I think make it stand out:

  • Open Source – http://qt.gitorious.org/qt-creator
  • Cross Platform – Works on Linux, Windows and Mac OS X like a charm
  • Actively developed and improved
  • Strong orientation and support for C/C++ native development
  • Excellent syntax highlighting
  • Magnificent code browsing and navigation in combination with useful and customizable keyboard shortcuts
  • Great and generic(works with any file you include, it doesn’t have to know anything special about it) code completion support
  • Easily customizable and Extensible through plugins
  • Support for integration with various Source Control Management tools like Git, Subversion, Bazaar, Mercurial, CVS and more.
  • Debugger integration with both GDB and Microsoft’s Debugging Tools for Windows through CDB (this one could use some improvement but it’s still better the debugging from the command line)
  • Custom configure, build, clean and deployment steps
  • Integration with tools like Valgrind
  • Per project settings
  • Vim editing mode
  • Code snippets
  • A pretty fast IDE, even when indexing hundreds of files with hundreds of thousands lines of code for the first time
  • Linux Man Pages context help integration – thanks to the flexible Qt Help System, this can be extended to almost any API

Although I only listed what I like the most on top of my head, Qt Creator has many cool and useful features and the only way to experience them all, is to start using it and you really should give it a spin, because as you can see, it’s feature list is nothing close to short. You can find some videos that show you some of the features I described above in action on Youtube, but keep in mind that they are kind of old(around 2009) and Qt Creator has very much evolved a lot since then.

The how to:

Now let’s get to the main purpose of this post, using Qt Creator with GNU Build System projects. A project that follows the GNU Build System usually is composed of a configure file and several other inputs(configure.ac, Makefile.am) that help in creating the Makefiles that will build the actual project on the target platform. The GNU Build System is, of course, much more then that, but for this tutorial, that’s all you need to know and because these steps will also work for projects that have only the Makefiles for e.g.

Next I’m going to show you how easy it is to use Qt Creator with the Nmap security scanner based on the steps outlined below:

  1. Download and install Qt Creator from http://qt.nokia.com/downloads. At the time of writing this, I’m using Qt Creator 2.2.1. I’m going to assume you’re going to be using Linux from now on, any distro should be fine as long as you are comfortable with it, however, I am using Debian(Squeeze). If you can’t make up your mind, just use Ubuntu. For Mac OS X and Windows steps should be similar.
  2. Decide on a project to use. I’m going to use Nmap since that’s what I’m working on currently and because it is the perfect example of a great open source project that’s low level enough to illustrate system programming using Qt Creator. You can download the latest version tarball from http://nmap.org/download(nmap-5.59BETA1.tar.bz2 is fine). After deciding on a project, you should install all of the development dependencies(the toolchain) that your chosen project needs, like the right compiler, libraries and other tools you might need. For Nmap, for e.g., we will need gcc, openssl libraries, libpcap, perl, python, etc. If you’re on a Debian like distribution(Debian, Ubuntu), luckily you can just do apt-get build-dep nmap and you get everything you need in one shot. If you do some tests builds from the command line, remember to clean the source tree before proceeding to the next steps, since you only want to add to the project only the base files and none of the auto generated files from the build system.
  3. Assuming you installed Qt Creator successfully, open it and go to File -> New File or Project -> Other Project and click on Import Existing Project. Advance to the next screen.
  4. In the current screen, give your project a name(that name will be used for your project files too so don’t go crazy with spaces or special chars) and select the location to your project source tree. In the nmap example, Nmap is a good project name and for Location browse to the path where you extracted the nmap source archive. Click on Next.
  5. The next screen, Project Management, is going to ask you if you want to use source control management(git, svn) for your project and if yes,then the project files will be the first to be added to the project. You should add these files to the scm repository only if you know that they would be usefull for others, otherwise you should keep them to your self, by using exclude directives like .gitignore files in git or svn:ignoreproperty in svn.  For the Nmap case, the project files are:
    • Nmap.config     – Project configuration settings like predefined Macros
    • Nmap.creator    – Qt Creator settings
    • Nmap.files         – A listing of all the files included in the project
    • Nmap.includes – Include directories
    • Nmap.creator.user – This one is excluded from adding to svn since it stores per user project settings.
  6. Assuming everything went fine, you should now have your project open in Qt Creator and ready to proceed to project settings. On the left side of Qt Creator click on Projects. Right now you’re looking at the Build Settings screen. This is where you set your project building and cleaning steps. For Nmap, the building process is made of the configure step and the make step. The clean process is made of the make cleanstep. Let’s add these steps to the build settings:
    1. On the Build Steps section click on Add Build Step -> Custom Process Step. Click on Enable Custom Process Step checkbox and add the following command to the Command: text box: ./configure. Move this step to the top of the stack by clicking on the upper arrow that fades in when you hover the build step. This step, the configure process, can only be run once and then disabled, since you don’t really need to run it again unless you run make distclean or modify the build system.
    2. Now expand the Make step and deselect the all checkbox in the Targets section. We don’t need that argument for Nmap, since Nmap’s Makefiles doesn’t have an “all” target. Now all the make step will do is issuing the make command.
  7. Now let’s proceed to the deployment steps, the Run Settings tab next to Build Settings, assuming that you would want to run Nmap in a basic form after a successful build just to know that everything runs smoothly. However for most cases, with tools like Nmap that have dozens of command line arguments you’re better of with running from an external terminal window, instead of modifying the arguments all over again in Run Settings when you need them changed. For this case, we’ll just want to run nmap with the version argument, ./nmap -V:
    1. In the Executable: text box input ./nmap.
    2. In the Arguments: text box input -V.
    3. Enable the Run in Terminal checkbox.
  8. Switch over to the Edit mode on the left sidebar and press Ctrl^Shift^s to save your changes and start Building by pressing Ctrl^b.

Wait for the build to finish and you are ready to run your executable by pressing Ctrl^r. In Nmap’s case you should see a window popping with Nmap version output:

That’s it! This is how simple it is to use Qt Creator with a project like Nmap, that has nothing to do with Qt or qmake whatsoever, all you need to do, is know a little bit about the specific project’s build process and toolchain and describe them to Qt Creator as build steps.

I’ve been using this process on many non Qt projects like John the Ripper, skipfish, VCMI(Heroes 3 oss clone), GNU Coreutils and many more besides Nmap, of course, projects which are in both C and C++ and all I can say is that Qt Creator helped me a lot in the cases where a text editor didn’t(and shouldn’t try to) excel at. None of the other choices available as free and open source IDE’s out there could match Qt Creator in my opinion and I really hope that the Creator will continue in the same manner it has so far, free and open source, cross-platform, actively developed and supported, flexible and not locked to the Qt framework. Here are other alternatives to Qt Creator that I’ve used in the paste and still use from time to time for specific certain tasks that they do well besides Qt Creator and adhere at least to the open source, cross-platform and good for native development rules:  KDevelop, Code::Blocks, Anjuta and Geany (no, I’m not going to mention Eclipse and Netbeans since they are not that good for native programming).

All in all, Qt Creator is a great IDE for general C/C++ development and even if it can’t currently beat the behemoth that is Visual Studio in certain aspects, it wouldn’t even be fair to compare them until Visual Studio will strive for cross platform compatibility, multiple compilers and debuggers support or the ability to use multiple revision control systems.

Give it a spin, you won’t regret.

Tagged with: