High Leverage Development

During our third month of porting SftpDrive to OS X, it became clear that creating and maintaining a cross-platform codebase of high performance network and filesystem code would be far more effort than we had hoped. It wasn’t that the project was impossible, or even absurdly difficult. It just wasn’t any fun. Every time I looked at a #ifdef WIN32, it was even more clear the code was becoming much more tedious to debug and maintain.

It seems our work on Slingshot spoiled us. A few months working with Ruby, Objective-C and C# left us feeling happy and optimistic about programming—anything was possible! Needless to say, the tens of thousands of lines of procedural C in SftpDrive for Mac no longer brought about the same feeling of joy. It seemed unfair that hotshotwebdevelopers, with their pretty MacBooks, got all the attention, and they got to use fun high-level languages like Ruby or Python. We were developing a truly useful piece of technology, but were stuck on Windows and spending more than 50% of our time dealing with “pedestrian” details like pointers, memory management, IRQLs, and IRPs.

Still, we fancied ourselves hardcore and kept at it even though it was hard (and sometimes boring). When it was time to write an auto updater for SftpDrive we spent hours upon hours searching on Google and MSDN trying to find a clean implementation that would work on a vanilla Windows 2000 installation. One option, WinInet was ridiculously ugly and verbose. Another option, WinHTTP, didn’t work on Windows 2000 GM. We ended up using libCURL. It was a ridiculous and frustrating waste of time.

We wanted to import httplib, and then just start making things happen. XKCD hits the nail on the head:

XKCD python

We couldn’t afford to keep spending time and energy writing software this way. Even if we could afford it, we didn’t want to spend our time this way. Web applications were being developed at an astounding pace in part because of centralized management and deployment (they never have to maintain different versions for Macs and PCs), but also because they were using modern interpreted languages. Web developers also used community-developed open-source projects when they needed some help on a routine problem. They didn’t have to reinvent the wheel at every turn, but instead focused on the core of their product. With high-level languages and good libraries, small teams can create great products at a rapid pace. We realized that we could write applications for the desktop in the exact same way.

We rewrote SftpDrive from top to bottom in Python, with a GUI in Objective-C. It’s called ExpanDrive, and it took 1/3rd the time that SftpDrive took to develop. We greatly leverage Python and and many open source projects—just like a web-developer. To minimize conflicts and to have the necessary control over the runtime environment, our build process extracts only the necessary bits from the full python distribution and packages it into the .app. We trim Python from 5000+ files to a few more than 400. Like many OS X apps, we use Sparkle.Framework to automatically distribute and install updates. We’re pushing out weekly updates which include more than just bug fixes. ExpanDrive has been a breeze to maintain and extend and the core remains perfectly cross platform.

Desktop applications aren’t dead, they’re just about to really get going.

  • komac

    yeah,recently I have ported an MFC app to Mac OS X,and it leads me to realize how important it is to write cross platform codes.

  • Emanuel

    Great post and an outstanding work!
    I’d also be interested in knowing how you made the trimmed down version of Python!


  • Pingback: Stranger Than Fiction » Computer Science Degree in a Blog Post()

  • http://blog.magnetk.com jonshea

    @Mark: I’m not at all jealous of people who spent their days working on cross browser compatibility. Ugh. Still, desktop applications

    @curios: That’s certainly in interesting idea, isn’t it?

    @Charles, Emanuel: We’re going to do followup blog posts about some of the tools and techniques we’ve used, including how to create a dependancy extracted subset of python.

  • http://www.wooji-juice.com/blog/high-leverage-why-yes.html Canis/Wooji Juice

    We’re doing something similar over at Wooji Juice, using Cocoa and Python to make commercial Mac apps. I’ve written a post exploring our reasons in more detail, over at our website. The short version? Impedance mismatch — or the lack of it.

  • Thomas

    I couldn’t agree more. People can be very scathing of higher-level languages but to me it’s about the enjoyment of programming, if I didn’t care about that I would try to write everything in pure assembly.

  • Pingback: » Blog Archive » MacBook Review()

  • Pingback: Random Non Sequitur » Blog Archive » ExpanDrive for president()

  • Pingback: ExpanDrivel » on C++()

  • http://none JoelB

    I too am interested in learning about the part of your build process that strips down Python from 5000+ files to 400. I would imagine that it mainly involves removing unused library modules … but it would be cool to see exactly how you guys do it.