Monthly Archives: February 2009

Pitfalls That You May Encounter when Running iPhone Unit Tests and How to Overcome Them

We have come to the last part of this series on iPhone Unit Testing. In this post we will cover pitfalls that you may encounter when running iPhone unit test and how to overcome each of them.

Pitfall: You may get this message in the Build Results: “Couldn’t register PurpleSystemEventPort with the bootstrap server. Error: unknown error code.

Overcome it by: This generally means that another instance of this process was already running or is hung in the debugger.”
If you get this, it just means that the iPhone Simulator is already running. Quit the Simulator and then run your tests again.

Pitfall: You get the error “Unable to send CPDistributedMessagingCenter message named SBRemoteNotificationClientStartedMessage to com.apple.remotenotification.server: (ipc/send) invalid destination port”.

Open up the “Tests” target by double-clicking on it, go to the Properties tab and clear out the Main Nib File text field.

Pitfall: XCode is not executing any of your unit tests. Instead it hangs at the line: “Running 1 of 1 custom shell scripts…”

Open up your Tests target, go to the Properties tab and clear out the Main Nib field. Do a Clean All. Then Build and your tests should run.

Since we discussed clearing out the Main Nib text field twice here, I think we should discuss why this would work. It will give us insight into how the Google Toolbox for Mac unit testing framework operates.

What I think it is related to is the Main Nib loading and that Nib having an App Delegate which implements applicationDidFinishLaunching. This sometimes causes everything to not work because GTMIPhoneUnitTestDelegate’s applicationDidFinishLaunching not to get called – and this is what runs the tests. Clearing out the Main Nib field will cause this potential collision to not occur and GTMIPhoneUnitTestDelegate will function properly by finding and running your unit tests.

Have you had run into any other pitfalls while doing iPhone Unit Testing? If so, feel free to leave a comment here. I also encourage you to visit the Google Toolbox for Mac discussion group on Google.

Getting Started Writing iPhone Unit Tests

Alright, well hopefully you have read the first two parts of this series. If not, then go back and read Is iPhone Unit Testing Possible? and How to Create an iPhone Project in Xcode That Can Run Unit Tests.

Make sure that you have an iPhone Project that is already set up for unit testing. You can either follow the steps in those last two posts or you can download a pre-made “example1″ iPhone Project that can run unit tests that I uploaded to GitHub.

In this tutorial, I will be showing you, the iPhone Developer, how to get started writing unit tests for your iPhone App. I will be doing this in the TDD style – where TDD == Test-Driven Development. In other words, we will write the tests first. You don’t have to do it this way, but I thought I’d show it to you since it is a nice practice that was ingrained in me from the Ruby on Rails community. This is not to say that it started in Rails – it really started back in the Smalltalk days with Kent Beck and the other original XPers (eXtreme Programming). But being involved in the Smalltalk, Java, Objective-C and Ruby communities in my career since 1995, I haven’t been seen such testing dedication as I have in the Ruby on Rails community. I hope it is something that really spreads in iPhone community as well.

So, how do we get started? Or in TDD-speak, how do we write our first failing test? You write failing tests first as sort of a TODO to yourself as a programmer. When you get it to pass, then you can mark off your TODO, but in a nice programmatic way.

You will need to create a Test Class. Then you’ll add a Test Method. This should reference a non-existent Class, which you’ll then proceed to flesh out. This is the classic Test Driven-Development (TDD) way of testing.

Create a new group with Project > New Group. Name this new group “Tests”.

Create a new Test Class with File > New File… and choose Cocoa Touch Classes / NSObject subclass.

Name your class “FooTest.m”. Make sure that it is only a member of the Tests target.

Run the tests now by pressing the Build button. Make sure that the “Tests” target is selected in the Active Target dropdown and “Simulator” is chosen a the Active SDK. Notice that the Test Class isn’t being included yet:

Test Suite '/Users/louie/builds/Debug-iphonesimulator/Tests.app' started at 2009-02-19 14:43:32 -0500
Test Suite 'SenTestCase' started at 2009-02-19 14:43:32 -0500
Test Suite 'SenTestCase' finished at 2009-02-19 14:43:32 -0500.
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite 'GTMTestCase' started at 2009-02-19 14:43:32 -0500
Test Suite 'GTMTestCase' finished at 2009-02-19 14:43:32 -0500.
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite '/Users/louie/builds/Debug-iphonesimulator/Tests.app' finished at 2009-02-19 14:43:32 -0500.
Executed 0 tests, with 0 failures (0 unexpected) in 0.001 (0.001) seconds

This Test Class doesn’t know it is supposed to contain tests yet. Let us change that.

Change the import statements to include “GTMSenTestCase.h” This is the Google Toolbox for Mac version of SenTestCase. This enables your test cases to be found by the GTM RunIPhoneUnitTest script. It also adds some additional capabilities we will cover later.

Change the superclass to be “SenTestCase.”

Your class header FooTest.h should now look like:

#import "GTMSenTestCase.h"


@interface FooTest : SenTestCase {

}

@end

If you run the tests now, you’ll see that FooTest is now being included:

Test Suite '/Users/louie/builds/Debug-iphonesimulator/Tests.app' started at 2009-02-19 14:44:30 -0500
Test Suite 'SenTestCase' started at 2009-02-19 14:44:30 -0500
Test Suite 'SenTestCase' finished at 2009-02-19 14:44:30 -0500.
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite 'FooTest' started at 2009-02-19 14:44:30 -0500
Test Suite 'FooTest' finished at 2009-02-19 14:44:30 -0500.
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite 'GTMTestCase' started at 2009-02-19 14:44:30 -0500
Test Suite 'GTMTestCase' finished at 2009-02-19 14:44:30 -0500.
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite '/Users/louie/builds/Debug-iphonesimulator/Tests.app' finished at 2009-02-19 14:44:30 -0500.
Executed 0 tests, with 0 failures (0 unexpected) in 0.002 (0.002) seconds

However, with 0 tests, FooTest is pretty boring. Let’s spice up FooTest. Switch over to FooTest.m. You can quickly do this with the keyboard shortcut Command + Option + up arrow.

Now add a Test Method. Test Methods are where the work happens with testing. They must start with “test” and the first character of “test” must be a lowercase “t” in order to be found by the test runner script.

Add this piece of code:

- (void)testBar {
	Foo *instance = [[[Foo alloc] init] autorelease];
	STAssertEquals(@"bar", [instance bar], nil);
}

Try to Build. You’ll get 3 errors. They all relate to Foo not being declared in FooTest. This actually is good! Well, in the classic TDD sense. What we have done is written what we want to happen when a client calls Foo and what the client expects back when it calls the bar method of Foo. Specifically, we want it to return the NSString @”bar”. This can also serve as good documentation for future programmers who are inspecting this code to see an example of how to use Foo.

Now let us fix these errors:

Select the Classes group.
Create a new Class with File > New File… and choose Cocoa Touch Classes / NSObject subclass. Yes, this is just like how we created a new Test Class. They are the same because we haven’t created a new template for a Test Class.

Name your new class “Foo”.

Make sure that it is a member of both of the targets “example1″ and “Tests”. The reason we want this is because Foo should exist in the main application but also should be available for testing. Going back, we can see that FooTest was only in the Tests target because we don’t want to ship our tests with the application. This will make it lighter and thus quicker to download when a user decides to purchase it from the App Store. Quicker downloads should mean a quicker path to riches! :)

Now try building again. Still 3 errors? Yes, because we need to make Foo known to FooTest.

Go to FooTest.h and add the line:

#import "Foo.h"

Now build again. You should have two errors now and four warnings.

Now Xcode tells you that “warning: ‘Foo’ may not respond to ‘-bar’.” Again this seems wrong, but this is actually a good thing with classic TDD. This just means that we need to declare the method bar. So let us do just that.

Go to Foo.h.

Add the method declaration:

- (NSString *)bar;

Go to Foo.m.

Add the method implementation:

- (NSString *)bar {
	return @"bar";
} 

Now build and you have success! This is known in TDD-speak as “green bar.” When you had failures due to errors/warnings, that was “red bar.” This had to do with the graphical indicators of the tools at the dawn of TDD. We can just call them “success” and “having test failures.”

This method is what I’ve used over the past year to help ensure good code quality in that iPhone apps I have built for my consulting clients. At this point, you know enough to be able to write unit tests, the iPhone way with the help of Google Toolbox for Mac. You can start to use more generic unit testing techniques that you may have used in other languages. There are also some iPhone-specific testing techniques that I will cover in the future if people are interested.

How to Create an iPhone Project in Xcode That Can Run Unit Tests

This is the second part of a blog series I am writing on How to Do iPhone Unit Testing or Lessons Learned from Unit Testing iPhone Apps.

I am targeting this at iPhone Developers.  My aim is to take you step by step through the process of setting up an iPhone Project in Xcode that lets you run unit tests.  There are other guides out there but I’d like to make this very visual and easy.

So let’s get to the question:

How do I set up an iPhone project that lets me unit test my app?

Step 1:  Create a New iPhone Project

Create a new project via File > New Project.  Choose “View-Based Application” (you can choose any, but we’ll use this type here.)  Name your project “example1″ to follow along with this tutorial.

After your project is created, it should look something like this:

 

The "example1" project after creation.

The "example1" project after creation.

 

Now that you have created the project, you can add the supporting files to enable unit testing.

Step 2: Create a Folder to Organize Your Unit Testing “Framework”

One of the lessons learned from my iPhone consulting is that iPhone projects can get big and messy.  It helps if you isolate your “frameworks” into folders in Finder.  I say “frameworks” loosely because you can’t really have Mac OS X-style frameworks in your iPhone projects.  There is no dynamic loading of libraries (which frameworks are) on the iPhone.  Instead, you end up copying all the files into your project.

In this example, we will create a folder named “UnitTestingFramework” in our project to separate it from the rest of our code.  If you’re using a lot of external code in your project, you could put this under another folder called “OpenSourceFrameworks.”  I’ve done this in most of my consulting projects, but we’ll keep it simple since we’re only working with one “framework” in this tutorial.

So: In Finder, create a new Folder inside your “example1″ project folder called “UnitTestingFramework.”

 

Create a folder named UnitTestingFramework in the example1 project folder.

Create a folder named UnitTestingFramework in the example1 project folder.

Drag and drop the “UnitTestingFramework” folder from Finder into the “example1″ project in Xcode.  When the sheet pops up, do not select “Copy items into destination group’s folder (if needed).”  The reason is because it already is in the “example1″ folder.

You should see a Group called “UnitTestingFramework” underneath the “example1″ Project.

 

You should see a UnitTestingFramework group underneath the example1 project.

You should see a UnitTestingFramework group underneath the example1 project.

Step 3: Find and Install a Unit Testing Framework

In the first post, I alluded to a group of “diligent and ingenious engineers”.  I was specifically referring to the Google Mac Team.  They’ve created a great project called Google Toolbox for Mac.  It actually does a lot more than unit testing.  For example, I used their Address Book interface to make working with Address Book on the iPhone much nicer.

In this tutorial, we’ll focus on one subset of Google Toolbox for Mac, the unit testing portion.  We won’t even use the entire unit testing portion, since there are helpers for Mac OS X unit testing as well.

First, you’ll need to download the source.  The latest stable version is their 1.5.1 from June of 2008.  You can get it from here: google-toolbox-for-mac-1-5-1.zip

You could also get the latest from Subversion, but we’ll focus on the stable version for this tutorial.

Unzip the file by double-clicking on it in Finder if it wasn’t unzipped already into a folder.  Then go into the folder and find the following 7 files:

GTMDefines.h

UnitTesting/GTMIPhoneUnitTestDelegate.h

UnitTesting/GTMIPhoneUnitTestDelegate.m

UnitTesting/GTMIPhoneUnitTestMain.m

UnitTesting/GTMSenTestCase.h

UnitTesting/GTMSenTestCase.m

UnitTesting/RunIPhoneUnitTest.sh

 

The 7 files you need to select from Google Toolbox for Mac.

The 7 files you need to select from Google Toolbox for Mac.

Drag and drop those 7 selected files to the “UnitTestingFramework” group in Xcode.  When the sheet appears, make sure to check “Copy items into destination group’s folder (if needed.)”  This time, we need to copy them from the Google Toolbox for Mac download to our project and specifically to the UnitTestingFramework folder we created in Step 2.

You have all the raw ingredients now, but you will need to perform a few more steps to be able to find and run unit tests in your iPhone project.

Step 4: Create a “Tests” Target

Create a new target via Project > New Target.

Choose iPhone OS > Cocoa Touch > Application.

Call it “Tests”.

In the first post of this series, we discussed that there are only two types of Targets available for iPhone projects.  Applications are the only ones that can run and we want to execute our unit tests.  That is why we are adding this new “Tests” target.

We are almost there.  Now we just need to tell the “Tests” target to find and execute our unit tests.

Step 5: Add a Run Script Build Phase to the “Tests” Target

Select the “Tests” target.

Control-click (or right-click or whatever finger gestures you use on your fancy new MacBook Pro with no buttons) and select Add > New Build Phase > New Run Script Build Phase.

Enter “./UnitTestingFramework/RunIPhoneUnitTest.sh” in the Script text area.  Note that if you had spaces in your folder, you would need to escape it with a backslash like “\ ” here.  So another lesson is to not include spaces in your folders to keep things simple.

 

Enter "./UnitTestingFramework/RunIPhoneUnitTest.sh" in the Script text area.

Enter "./UnitTestingFramework/RunIPhoneUnitTest.sh" in the Script text area.

What is this “RunIPhoneUnitTest.sh”?

It is a shell script that the Google Mac team has created.  It sets up some environment variables to help with executing the unit tests.  The real magic will be explained later in this series.  Hint: it is in those files you selected and dragged into your project.

Step 6: Ensure All the “UnitTestingFramework” Files Are Part of the “Tests” Target

Select the “Tests” target in the Active Target portion of the Toolbar.  You can also select in via the menu item Project > Set Active Target > Tests.  Ensure tha the Active Build Configuration is Debug and the Active SDK is Simulator.

Select the UnitTestingFramework group.  Look at the list on the right side and make sure that all the *.m files are checked.  This ensures that they will be included in the Tests target.  Check any that are unchecked.

Note that you’ll want to make sure these are all unchecked in your “example1″ target as well.  You don’t need these testing “framework” files in your regular application target.

 

Check all the UnitTestingFramework *.m files for the Tests target.

Check all the UnitTestingFramework *.m files for the Tests target.

 

Step 7: Run the Tests and Verify Success

At this point, the project is fully set up to run unit tests for an iPhone project.  To verify this, press the Build button on the toolbar or use the menu item Build > Build.

Open up the Build Results so you can see the progress.  You can use the menu item Build > Build Results.

To better see the progress, you will want to look at the detailed Build Transcript.  To see this, you’ll need to click on the little transcript icon in the Build Results window.

 

Click on the Build Transcript button to see the details of the unit tests progress.

Click on the Build Transcript button to see the details of the unit tests progress.

You should see: (give or take a few miliseconds)

“Executed 0 tests, with 0 failures (0 unexpected) in 0.004 (0.004) seconds”.

This is success!  Your project is set up correctly.  We have no tests yet.  We will cover that tomorrow.  But this shows that the unit testing “framework” is running correctly.  If we did have any tests, they would execute and show in the Build Details.

If you don’t see this, don’t worry – try reading part 4 of this series: Pitfalls That You May Encounter when Running iPhone Unit Tests and How to Overcome Them.  You can also visit the Google Toolbox for Mac Google Group for support via an email discussion list.

Note that the results were reported twice – this is a known bug in 1.5.1 and is fixed in a later version.

You can see the completed version of this project at http://github.com/luisdelarosa/iphone_unit_testing.  That’s sort of like when you see a chef pull out a fully cooked pie at the end of  cooking show.

If you have any questions about this tutorial, let me know in the comments.

Is iPhone Unit Testing Possible?

Lessons Learned: Unit Testing iPhone Apps

This is the first part of a four part series on How to do iPhone Unit Testing. You could also call it Lessons Learned from Unit Testing iPhone Apps. I’ve learned a lot of lessons over the past year doing iPhone Consulting and I want to share them with everyone.

Here is the schedule for this series:

  1. Is iPhone Unit Testing Possible?
  2. How to Create an iPhone Project in Xcode That Can Run Unit Tests
  3. Getting Started Writing iPhone Unit Tests
  4. Pitfalls That You May Encounter when Running iPhone Unit Tests and How to Overcome Them

This is targeted at iPhone Developers, especially those who have done unit testing in other languages and frameworks.

A bit of backstory: My name is Luis de la Rosa and I am an iPhone Consultant. I have been making iPhone applications for clients the past year through my company Happy Apps LLC. I have also been developing Mac OS X apps and Rails apps for the past 3 1/2 years and in other languages for 14+ years now.

To start off, you might ask: Is iPhone Unit Testing even possible? I don’t see it as an option when creating a New Target in Xcode like I can with Mac OS X applications.

Apple says it is not possible. In the “Xcode Unit Testing Guide”, it says

“iPhone OS Unit Testing Support: Unit tests are not supported for iPhone applications.” 

But what this really means is that Unit Test Bundles, which are dynamic, are not allowed on iPhone because dynamic bundles of all kinds are not allowed on iPhone. So the normal way of adding Unit Tests to an Xcode project is not available to you. (iPhone projects are Xcode projects.)

However, there is a way to add unit tests to an iPhone project!

To understand why, you need to understand the two types of targets available to an iPhone project. These two are:

  1. Application
  2. Static Library

To put it another way: the normal Unit Test Bundle target for Mac OS X applications is not available because there are no Dynamic Bundle targets. A Static Library target cannot be executed and we need some sort of execution in order to run the tests. So we need to somehow use an Application target to run our unit tests.

The Application target will need to do the following things:

  1. Find all the unit tests in your Project
  2. Run all the unit tests
  3. Report the results of running the unit tests. This should include the number of successes and failures overall and also the results of each individual test.

You could do this from scratch, but fortunately there are already some diligent and ingenious engineers out there who have already done the work for you.

In tomorrow’s segment: How to create an iPhone Project in Xcode that can run unit tests.

If you found this helpful and/or interesting (hopefully both), please leave a comment.

C4[2] t-shirts available

I love to wear Tech Conference t-shirts.  I should really take a picture of them all sometime.  They help get me into a productive mood.

Wolf put up an official C4[2] t-shirt store.  It was a pretty interesting design but it didn’t have words so my wife was really confused as to where I got it.  She thought it was some sort of gardening thing.

Jose on the other hand, studied the t-shirt, analyzed the logic and found an optimization.  Gotta love the geekiness in that.

Now if I could only get old WWDC t-shirts too.  I left the 2008 one at the hotel and my other ones are getting worn out.

Update (Feb. 19, 2009): You can now buy The Wolf-Rentzsch t-shirt!  I was sitting right near bobtiki and josevazquez when they were making this shirt on the Southwest flight to C4[2].

I’ve been busy

It has been a few months since I’ve blogged here and where have I been? Short answer is that I, Luis de la Rosa, have been busy. I’ll get into the details over the next series of posts.

In terms of programming, I’ve been mostly doing Objective-C the past few months. I’m still doing Ruby on Rails, but Cocoa and Cocoa Touch are my main focuses. I actually did get paid to do some Erlang as well. Finally I joined an open source project that I’ve used on most of my iPhone projects. So there’s lots to talk about in the near future.