iPhone Programming Tips: building Unix software

The NDA that was part of the agreement between Apple and registered iPhone developers was lifted yesterday. Not an hour had gone by before other developers started to share tips, code or even full frameworks. Like them, we'd also like to contribute some of the learnings we've found during our work with Apple's SDK.

We've decided, therefore, to write a second piece in our internationally acclaimed iPhone Programming Tips series. Our first article, devoted to image orientation techniques, was written during our jailbreak apprenticeship, before the NDA even existed - how's that for anticipation?

This time we'll explore a very different area: how to build a Unix source package in such a way it can be used as part of your iPhone project.

The Problem

So you've found this wonderful open source Unix package that perfectly handles one of the areas you need in your application. Being a publicly scrutinized, well-maintained library, you decide you would waste your time if you tried to do something similar, so you try to use it in your project and concentrate on your own features. Or maybe you simply want to test how this thing would work in the iPhone. Whatever your reasons are, you download the distribution and read the INSTALL file. This package is so well designed it compiles under a zillion Unix variants, including many evolutionary dead ends. It includes a fancy "configure" script that guesses everything it needs to know to configure itself properly. Apple's Developer Tools include a full gcc/make toolchain, so you perform the sacred ritual "./configure; make" and presto, you obtain an Intel dynamic library ready for use!

Wait a minute, that's not what you need. You want to cross-compile for ARM so that it runs on the device, of course. And you need it to run in the simulator, too.

Being a resourceful, tough programmer, you drop the source code on your Xcode project, alongside the rest of your code. You build your project and get like 148 errors and 57 warnings. Naturally, there are lots of definitions that ./configure should have defined, and the source defaults must be those of a PDP-11, or something. You try to tweak the settings manually for 10 minutes before you give up.

Your next step is to tell the configure script to cross-compile for a different platform. But you don't know exactly how to tell it your platform is like Mac OS X, only running on an arm chipset, what's the problem about that. After reading the scripts and template files and googling your way around, you use something like ./configure --host=arm-apple-darwin. This doesn't work, either, because configure insists on using your standard Mac OS X system libraries and headers instead of the ARM ones for the iPhone. You then try to tell configure to use the gcc compiler in your /Developer distribution, and this still doesn't work. Oh, well. You know what you need to do - investigate compilation options and library locations - but it's painful.

Wouldn't it be nice to have some notes on how to set up your environment to compile an Unix package from the command-line, but using Xcode's development libraries for the iPhone?

The Solution

If you have read the introduction above, you will have noticed that it is just a feeble attempt to disguise the fact that this article is only going to tell you how to configure your environment variables to compile a Unix library using Xcode's toolchain from the command line. That's it. It is admittedly not a very glamourous or innovative task; however, we still had to devote a few lenghty hours to set everything up properly, following the embarrassing steps outlined above, one after the other.

So, without further circumlocutions, let us show the damned variables. I'll dissect one of the build scripts I have actually used, accompanying the definitions with some comments or caveats.

The following two definitions point to the root of the command line developer tools and iPhone SDK. You may need to modify them to update the location in your own system, or if you are using a newer version of the SDK:

export DEVROOT=/Developer/Platforms/iPhoneOS.platform/Developer
export SDKROOT=$DEVROOT/SDKs/iPhoneOS2.0.sdk

Next, let's save the current build environment - we'll use it later to build the i386 version of the library which will run in the simulator.

# Save relevant environment
U_CC=$CC
U_CFLAGS=$CFLAGS
U_LD=$LD
U_LDFLAGS=$LDFLAGS
U_CPP=$CPP
U_CPPFLAGS=$CPPFLAGS

We'll now define the values we need to use to target the ARM architecture. Compilation flags in my case look something like this:

export CPPFLAGS="-I$SDKROOT/usr/lib/gcc/arm-apple-darwin9/4.0.1/include/ -I$SDKROOT/usr/include/"
export CFLAGS="$CPPFLAGS -arch armv6 -pipe -no-cpp-precomp -isysroot $SDKROOT"
export CPP="/usr/bin/cpp $CPPFLAGS"

Linking flags are a bit more tricky. You need to ensure the output of the compilation process is a static library, and not a dynamic library or an executable file. Dynamic libraries can in fact be produced, but the iPhone sandbox will sadly refuse to load them at runtime - only dynamic libraries and frameworks in predefined system locations can be used. There's a mention about this limitation somewhere in the Development Agreement.

Even though you won't be using dynamic libraries in your project, some packages are configured in such a way that it is easier to let them compile the dylib then ignore it, rather than trying to convince them not to create the dynamic code. If you encounter such a case, you should configure your linker in a way similar to this:

# dynamic library location generated by the Unix package
LIBPATH=src/.libs/<libname>.dylib
LIBNAME=`basename $LIBPATH`

export LDFLAGS="-L$SDKROOT/usr/lib/ -Wl,-dylib_install_name,@executable_path/$LIBNAME"

This will create a valid dylib file that you will be able to use within Xcode; however, it won't run in the iPhone as described above.

Therefore, you will actually be interested in using the static library version, so we'll store its location:

# static library that will be generated
LIBPATH_static=src/.libs/<libname>.a
LIBNAME_static=`basename $LIBPATH_static`

Now we are ready to run the configure script and build the libraries.

./configure CC=$DEVROOT/usr/bin/arm-apple-darwin9-gcc-4.0.1 LD=$DEVROOT/usr/bin/ld --host=arm-apple-darwin
make

Depending on the package you are trying to compile, you might need to supply additional arguments to the configure script. Some packages, for example, will accept arguments indicating whether a static library or a dynamic one should be built. It is also frequent to disable features or modules you know you won't use in your project. You need to refer to your package documentation for fine tuning details.

After make finishes (hopefully without errors), we'll move away the generated libraries to a safe location:

mkdir -p lnsout
cp $LIBPATH_static lnsout/$LIBNAME_static.arm

We repeat now the same steps, but targetting the i386 architecture. This will allow us to build libraries compatible with our iPhone simulator environment.

make distclean

# Use default environment
export CC=$U_CC
export CFLAGS=$U_CFLAGS
export LD=$U_LD
export LDFLAGS=$U_LDFLAGS
export CPP=$U_CPP
export CPPFLAGS=$U_CPPFLAGS

# Overwrite LDFLAGS
# Dynamic linking, relative to executable_path
# Use otool -D to check the install name
export LDFLAGS="-Wl,-dylib_install_name,@executable_path/$LIBNAME"

# ToDo - error checking
./configure
make

# Save generated binaries
cp $LIBPATH_static lnsout/$LIBNAME_static.i386

After we have produced the i386 and arm versions of our library, we will create a fat Universal Binary enclosure containing both of them:

# Create fat lib
$DEVROOT/usr/bin/lipo -arch arm lnsout/$LIBNAME_static.arm -arch i386 lnsout/$LIBNAME_static.i386 -create -output lnsout/$LIBNAME_static

Finally, you can add the generated library and any necessary development header files to your project and build it. If everything went well, your application will be linked with the library and will run correctly in both the simulator and the device.

As explained above, this is mostly basic Unix tinkering, but it took us a while to get the right configuration. Maybe we were just rusty, so if you know a better way to achieve this, please do let us know!

WWDC 2008

If I had to summarize this year's WWDC in one word, it would be "Awesome". A year ago, we went to WWDC with no clear goals. We attended a lot of sessions, and met a few people. Overall, it was a good experience, in which we gained a lot of technical knowledge.

This year, it was different. Our primary goal was not to spend all of our time in sessions. We wanted to make our visit business oriented, meeting people, and getting feedback about different aspects of our applications.


After 2 years of working with him, we finally met Louie Mantia, our graphic designer. Currently, he works mostly for our friends at Tapulous, but, fortunately, he still does some work for LateNiteSoft.

We also met Chuck Soper, Cabel Sasser, Daniel Jalkut, Mike Lee, Kenichi Yoshida, Steve Scott, Tristan O'Tierney, Colin Wheeler, Kevin Hoctor, and some others in the SF Mac Indie Soiree.

During the week, we met with many Apple employees, as well as other entrepreneurs, developers, designers and journalists. We participated in a few labs and different consultancy sessions. We were present at the parties and events, but overall we probably did not make it to more than 5-6 sessions each.

The overall result has been very good, and we are very satisfied. Within the next few weeks we hope to start showing you the outcome of our visit.

Dark Xslimmer's Toolbar

A few days back, I watched Cabel Sasser's C4 presentation. I thought it was a very nice presentation, with some insightful bits that can only be produced out of experience. I particularly enjoyed the part where he explains why they had to create their own toolbar. We had to do the same.

During the summer of 2007, Louie Mantia challenged us to produce a black themed version of Xslimmer. As Leopard approached, we were thinking that providing Xslimmer with face lift was a good idea. So, making Xslimmer black would made it look nicely different. At that time, it did sound as something simple: just make the grey areas of the window become black, and the black ink become white. Easy.

I started developing the window part. Fortunately, some people had walked that path before me. Making the window black was relatively easy. Using a background image for the window title + toolbar area was the only tricky part, as, in the beginning, it was not correctly rendered. The top half of the image was drawn in the bottom half of the toolbar, and viceversa. Fortunately, I discovered that setting something called "the pattern phase", within the graphics context, cured the wound. The black window was done.

Next step was to make the toolbar font white. I checked the documentation. Nothing. Checked Cocoa development sites. Nothing. Checked everywhere I could think off. Nothing. Asked other developers. Nada. How could it be? Similarly to the guys in Panic, who were fighting to get access the 3 bottom pixels of NSToolbar, we just wanted to draw white text with a bit of grey shadow, and make the black dotted lines, white dotted lines. That was all. I tried every hack possible, subclasses, all sort of undocumented stuff to try to get access to the text rendering part, so I could invert the colors. Nothing.

My first thought was: "I will just build the toolbar using Interface Builder". Then, I started to design how it would work and operate. I soon realized that it was not going to be an easy task. Not only would I had all sort of issues placing the different pieces of the toolbars, but also controlling them during run time. In addition, Xslimmer already used toolbars in 4 different places, so the code to manage them was already there and would need major rework.

I decided it was time to create our own toolbar management system, a system that would operate in the same fashion as Apple's, but that would provide the needed flexibility. This would allow us to not change the existing toolbar management code, and, at the same time, make the text color any color we wanted. It took a while, but it was fun, and the results paid up:

It was time to get other people's opinion. So, during our beta testing phase, we had a poll. Most of our users did not fully like the dark theme. We held several internal discussions and finally went back to a more "Leopard-like" look, the one that Xslimmer uses today. Given the fact that code for toolbar management was unchanged, rolling back was trivial.

So, at the end, we never used the newly developed toolbar system. At that time, Chris Messina suggested that there might be some interest out there in continuing its development, that we should open source it. To make the toolbar system open source would imply some work: I would have to extract the toolbar and window code, produce a sample application, etc. So, prior to making the effort, I was wondering: Would anyone be really interested?

iPhone programming tips: image orientation

One of our main focus when we started to write Sketches was to leverage as much as possible the existing technologies already available in the iPhone. This is not a new idea: just as all Mac OS X apps cooperate with each other, we thought that integrating Sketches with the rest of apps in the system would make it a far more fun and pleasing app.

Given the complete lack of documentation or experience, trying to figure out how the APIs work and fit together with each other was no easy endeavor. However, we bravely set out to study the Cocoa iPhone APIs, armed with our previous knowledge of desktop Cocoa, and a lot of patience. Sometimes we thought the effort might be overkill, but being perfectionists at heart, we were uneasy to deliver something that we ourselves considered suboptimal. We are satisfied with many of the results we got, and people seem to appreciate them too. In fact, we've had several fellow coders ask us about details in our image handling code, or how we manage to send emails from Sketches without actually launching MobileMail.

One of the areas we are frequently asked about is the way we handle images shot from the camera or picked from the photo album, and how we detect their orientation. Instead of answering the questions privately, we thought it could be interesting for others to post them here. Maybe our explanations will become outdated next week after The Event - if so, that'd be great news for all :)

In the following paragraphs I'll try to describe some of the interesting "features" I found when guessing how to use the Camera and Photo Album APIs, with a special focus on detecting orientation. I may be utterly wrong in my interpretations, but so far they have been working reasonably well for Sketches :)

* Warning: technical burble ahead *

The first idea that comes to mind if you want to have the camera shoot pictures and use them in your app, is to use the cameraController:tookPicture:withPreview:jpegData:imageProperties callback invoked by the CameraController class. It looks promising, but after some experimentation, we found this approach was not as flexible as we required. Instead, we are using a different mechanism in Sketches. We subscribe to the CameraImageFullSizeImageReadyNotification event, which is sent just after a new photo has been taken. Early in our development we decided to subscribe to all notifications and log them in a file, a simple technique that was instrumental to learn about the existence of this particular notification. In our code, the method that receives the notification is similar to the following:

- (void) cameraImageFullSizeImageReadyNotificationObserver: (NSNotification *) notification
{
    NSLog( @"cameraImageFullSizeImageReadyNotificationObserver" );
    [cc stopPreview];
    CameraImage * cameraImage = (CameraImage *) [notification object];
    if ( cameraObserver && [cameraObserver respondsToSelector: @selector(imageWasSelected:)] )
    {
        [cameraObserver performSelector: @selector(imageWasSelected:) withObject: cameraImage];
    }
}

The important piece of information here is that the "object" property of the notification contains an instance of the CameraImage class, which belongs to the PhotoLibrary API framework. In a moment we'll show how to deal with CameraImage instances.

For photos picked with the photo album browser, we use the PLUIController class. A line like

[[PLUIController sharedInstance] setDisplayingPhotoPicker: YES];

will trigger selection of a photo from the photo album. If you set a delegate object, the photo album will invoke your delegate's imageWasSelected method, indicating as a parameter the CameraImage instance that corresponds to the image selected. Note we are using the shared PLUIController instance and do not instantiate a copy ourselves - results are more difficult to achieve the other way around, because the constructor of the shared object actually knows how the instance has to be instantiated, while we have no idea about the properties we should set to make it work.

Therefore, for both types of photo selection (shooting with the camera or selecting a photo from the photo album), we end up with a CameraImage instance to deal with. A CGImage can be created from the CameraImage, using the following API call:

int orientation;
struct CGImage * cgimage = [image createFullScreenCGImageRef: &orientation];

Now we have a Core Graphics image, which is a good thing because there's actually a whole lot of documentation about that.

However, this is where things get a bit hairy regarding how to determine image orientation.

First of all, there are three different sets of values that refer to orientation characteristics:

- If you copy photos from your iPhone to your computer and look at them in an EXIF browser, you'll see that they contain one of the following values: "1" for "normal" images; "3" for images that are rotated 180 degrees; "6" for images that are rotated 90 degrees; and "8" for images that are rotated 90 degrees counter-clockwise.
- The orientation value returned in the integer above does not correspond to the EXIF values just mentioned. The values returned by createFullScreenCGImageRef are, respectively: "1", "2", "3", "4". To make things amusing, images shot with your phone in the vertical upright position will typically be stored to disk with a 90 degrees CCW rotation; therefore, the orientation value you'll receive if you pick such a photo from the photo album will be "4".
- Third, the device orientation, as reported by UIHardware, uses a different set of values. They are the following: "1" (phone vertical, upright position); "2" (phone upside down); "3" (phone rotated 90 degrees CCW with respect to the vertical position); "4" (phone rotated 90 degrees CW).

Therefore, after you select a photo from the photo library, you should check the orientation value you get when creating the CGImage reference, and then you have to correct the rotation using the rotation and translation transformations provided by the CoreGraphics API.

For photos shot with the camera, however, the orientation value returned by createFullScreenCGImageRef at that particular callback is always "4", no matter what the device orientation is, or the way the photo is stored in the disk. This probably happens because my initialization of the camera related APIs is possibly incomplete, but I haven't figured out how to achieve a better result. For shots, therefore, what we do is use the device orientation value, then rotate the photos according to the description above.

Another option I tried was to open the physical files that refer to the CameraImage and read their properties. I was hoping to get the EXIF orientation value, but it seems that at this stage not many properties are available. In fact, code like the following yields a dictionary with just a "FileSize" property:

NSString * imgPath = [[image fileGroup] pathForFullSizeImage];
CGImageSourceRef imageSource = CGImageSourceCreateWithURL( (CFURLRef) [NSURL fileURLWithPath: imgPath], NULL );
NSDictionary * imageProps = (NSDictionary *) CGImageSourceCopyProperties( imageSource, NULL );

We are using these techniques in the current version of Sketches, and will also be the basis for some new features we are still cooking up. We hope they are useful to other iPhone programmers too!