XING Devblog

Keep It Functional – iPhone Test Automation

| Posted by

Currently all the XING Mobile Apps generate more than 20% of the overall XING traffic and this number will likely increase in the near future. In my last devblog post, I dealed with Android Apps and how to implement automated tests with the framework Robotium. This article describes how to implement automated tests using the tool KIF (Keep It Functional).

KIF is an open source test framework developed by the company square. It is an iOS integration test framework that allows you to implement test cases with objective C that can be executed, currently only, against the iPhone/ iPad simulator. The KIF tests are integrated to your Xcode workspace, there is no need for additional servers or services that must be started.
KIF is like a Grey Box Test Tool because you need knowledge of the structure of the App you want to test. KIF use the provided accessibility labels from iOS to interact with the UI elements in the App. The labels are normally used to make Apps available for people with visual disabilities.
With KIF you are able to simulate real user interaction like tap, click, drag, enter text and many more. Also you are able to verify elements on the screen. It is really fast in executing the test cases on the simulator. Also it is really easy to integrate to your xCode and Continuous Integration environment.

Don’t submit your tests to Apple

It is really important that the KIF tests are not part of your production code, otherwise Apple will deny your submission because of undocumented APIs. Why this?
KIF uses these undocumented Apple APIs, to interact with the App, but that is not a problem when it comes to developing the tests. Most of the available iOS test frameworks use these undocumented Apple APIs.

Installation

To install and configure KIF please follow the provided information on the github page. These instructions are always up to date to the latest version of KIF.

Accessibility Inspector

If your project is configured with KIF you have also to activate the “Accessibility Inspector” on the iPhone or iPad simulator to have access to the labels. Do the following steps on the simulator:

  1. Open the Settings
  2. Click on General
  3. Click on Accessibility
  4. And activate the Inspector

Afterwards you see the inspector with a colorful layer within your simulator. You can drag the layer wherever you want. Clicking the small (x) on the left side opens the inspector. If you now do a single tap on an app you get information about the accessibility label and some more information (Screenshot). If you do a double click you can e.g. enter the app. If you close the inspector layer with the (x) again, you have again single click on the simulator.

NOTE: The inspector must be activated to use KIF for testing. Now you can start implementing your test classes.

iPhone Simulator with Accessibility Inspector

Code Example

KIF has three main classes that can be used to implement tests within your project. There is a test runner the KIFTestController, a scenario the KIFTestScenario and a step the KIFTestStep. The KIFTestController is like a test suite where all the test scenarios are listed that should be executed. A KIFTestScenario is like a container that includes several steps that should be done within a test. A step is the smallest and simplest action that represents user interaction like tap, enter text or wait for views.

The following listings show the Controller h-file and m-file. The m-file includes the list of possible scenarios.

XINGTestController.h

#import
#import "KIFTestController.h"
@interface XINGTestController : KIFTestController {}
@end

XINGTestController.m

#import "XINGTestController.h"
@implementation XINGTestController
 
   - (void)initializeScenarios {
      [self addScenario:[KIFTestScenario scenarioLoginWithWrongCredentials]];
      // Add here additional scenarios
   }
@end

Now you need to implement the scenario scenarioLoginWithWrongCredentials. You need again an h-file and an m-file for that. The h-file includes the scenario definitions. The m-file is the class where the test methods are implemented.

KIFTestScenario+XINGLogin.h

#import
#import "KIFTestScenario.h"
 
@interface KIFTestScenario (Login)
 
   + (id) scenarioLoginWithWrongCredentials;
 
@end

KIFTestScenario+Login.m

#import "KIFTestScenario+Login.h"
#import "KIFTestStep.h"
 
@implementation KIFTestScenario (Login)
 
   + (id)scenarioLoginWithWrongCredentials{
      KIFTestScenario *scenario = [KIFTestScenario scenarioWithDescription:@"Wrong credentials"];
 
      // enter wrong credentials in the username and password field
      [scenario addStep:[KIFTestStep stepToEnterText:@"wrongusername" intoViewWithAccessibilityLabel:@"Username"]];
 
      [scenario addStep:[KIFTestStep stepToEnterText:@"wrongpassword" intoViewWithAccessibilityLabel:@"password"]];
 
      // click the done button on the keyboard
      [scenario addStep:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"done"]];
 
      // verify that the error message is shown with a localized string
      [scenario addStep:[KIFTestStep stepToWaitForViewWithAccessibilityLabel:LocalizedString (@"FAILED_MESSAGE")]];
 
      // click the button on the error message
      [scenario addStep:[KIFTestStep stepToTapViewWithAccessibilityLabel:LocalizedString (@"OK_BUTTON")]];
 
      return scenario;
   }
 
@end

Now you have a really simple test that tries to login with wrong credentials against an app. After pressing the done button on the keyboard KIF verifies with the stepToWaitForView… method that the right error message is shown. The following screenshots show the login screen where KIF tries to login with the wrong credentials. Please also have a look at the provided screencast videos at the end of this article. You get a really good impression how KIF is working.

   

Continuous Integration

It is highly recommended to integrate KIF to a CI server like Jenkins to build up a regression testsuite to be sure that the existing functionality is still working after a commit. To integrate and execute your KIF tests from the CI server you must be able to start the simulator from the command line. To do this, you need an additional tool called WaxSim. One Note from the KIF github page: “Note that the Square fork of WaxSim provides a number of bug fixes and some useful additional functionality. Your CI script should resemble something like the this:”
Adapt the following script to your environment and add it to your Jenkins build configuration:

#!/bin/bash
 
killall "iPhone Simulator"
 
set -o errexit
set -o verbose
# Build the "Integration Tests" target to run in the simulator
# xcodebuild -target "Integration Tests" -configuration Release -sdk iphonesimulator build
 
# Run the app we just built in the simulator and send its output to a file
 
# /path/to/XINGApp.app should be the relative or absolute path to the application bundle that was built in the previous step
/path/to/waxsim -f "iphone" "/path/to/XINGApp.app" > /tmp/KIF-$$.out 2>&1
 
# WaxSim hides the return value from the app, so to determine success we search for a "no failures" line
grep -q "TESTING FINISHED: 0 failures" /tmp/KIF-$$.out

Store this script as a shell script within your KIF test folder and call the script from the CI server. The iPhone simulator should start automatically and should execute the tests on the simulator.

That is the way, how we at XING use KIF to automate our iPhone App. Besides all the manual testing we use the automated tests to check, that code changes doesn’t affect to current functionality.
We hope that this post gives you an impression on how to build a test automation suite for your iPhone or iPad apps!

Screencasts

The following screencasts show the functionality of KIF. The first video shows the example code mentioned in this blog post. The second video shows how KIF is starting the XING handshake with some testusers on our testsystem.

Login with wrong credentials

Handshake

The latest XING iPhone App is available here.

Further information can be found in this really good google group about KIF.

This post based on sources from:

About the author

Daniel KnottDaniel Knott works as a Manager Quality Assurance in XING’s mobile team. He likes to automate test cases using technologies such as Robotium, KIF(Keep It Functional), Selenium and Java.
XING Profile »


4 thoughts on “Keep It Functional – iPhone Test Automation

  1. Nice write up. I’m a big fan of KIF for teams where developers write the tests, or the QAs are also programmers.

    I found some code that let’s you enable accessibility on the simulator programmatically when the app starts (KIF target only, obviously). It enables accessibility without actually showing the inspector, which is handy for your CI box.

    http://sgleadow.github.com/blog/2011/10/14/enabling-accessibility-for-ios-applications/

    I have also managed to use a variation of this code to enable accessibility on the device http://sgleadow.github.com/blog/2011/11/16/enabling-accessibility-programatically-on-ios-devices/, but the original author of the code believes having accessibility on the device causes performance problems for table cell reuse and things.

  2. Hi/Hello,

    i am IOS Apps developer new to automation testing .your suggestions will help me out from this, Thanks in advance
    Please give the answers for the following questions
    1.Does it require IOS application Source Code.
    2.will it work on Apple device such as I phone,I pod and I pad
    3.How do i can integrate on Xcode5 and IOS 7

    Thanks
    Upendra N

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>