XING Devblog

Robotium @ XING. Automated regression tests on mobile Android devices

| Posted by

As mentioned in a previous blog entry, the Quality Assurance department at XING has a strong focus on test automation. Every code change or new functionality could affect our existing features and their behavior. As there is never usually enough time for manual regression testing, which is also highly inefficient, we have always used Selenium, TestNG and Java to perform automated regression tests which are in turn limited to XING’s web platform.

According to a new forecast from BITKOM, more than 10 million smartphones will be sold in 2011 in Germany alone. This increase in devices goes hand in hand with a rise in mobile internet usage. So it is really important for XING to have high-quality apps in the respective app stores so as to provide the customers with access to the platform while out and about.

To fulfill these requirements, XING has a team dedicated to mobile applications such as the iPhone app, the Android app and the mobile web app touch.xing.com.

The high level of diversification of Android devices poses a special challenge for the Quality Assurance team as the market is fragmented with a number of different vendors and their customized user interfaces. There are various Android software versions that can be installed on devices like smartphones or tablets, and the XING app needs to cover all of them.

As things stand, the XING Android app is used by customers running Android versions 2.1, 2.2 and 2.3, which are mostly installed on the devices HTC Desire/ HD, Samsung Galaxy S, Galaxy Tab, HTC Legend, HTC Wildfire, Motorola Milestone and LG Optimus.

Another requirement that affects quality assurance is the hardware performance of the devices. Just because the app works well on current devices like the HTC Desire HD does not mean that the app will also work well on older devices with lower performance such as the HTC Legend.

Besides the problems with software and hardware, another parameter makes app testing a lot more complex – language. The app could be used in different languages and all text resources must be correctly translated into these languages. Text resources also have to fit on devices with differing screen sizes.

These three factors, i.e. software, hardware and language, make it impossible to manually test every code change on the devices with different Android software versions and language settings as the amount of work involved is simply too high.

To solve these problems, automated regression tests are necessary to deliver an app with good quality that runs on every device with various settings. Up to now, the XING Android app was tested using automated unit and manual functional tests.

Robotium

The Robotium framework is used to develop a regression test suite for the XING Android app. Robotium is a “Black Box” testing tool that is able to simulate and automate user interaction such as touching, clicking, text entry and any other gesture that is possible on a touch device. The tests could either be executed on the Android simulator (AVD – Android Virtual Device) or on a real device. Executing such tests on real devices has the major advantage that the app is running on real hardware within a real environment so potential performance problems can be identified at an early stage with this technique.

Robotium is built on the Java programming language and the JUnit test framework. As mentioned before, Robotium is a “Black Box” testing tool so you don’t need any further information about the Android app’s structure or implemented classes. All you need is the name of the main class and the path that links to it.

To develop stable and reliable tests, Robotium offers many methods that react to different graphical elements within an Android app, such as.

  • clickOnText(“Secure Login”);
  • clickOnButton(“Save”);
  • searchText(“Logout”);
  • goBack();
  • getButton();
  • isRadioButtonChecked();

With these simple methods, robust automated tests can be implemented really quickly. By combining them with JUnit, you then have additional ways of checking values or user interactions on the device for the correct response, which in turn makes these tests even more powerful.

Getting started. What’s required?

The following software is required to implement automated tests for Android apps:

  • Eclipse-IDE
  • JAVA SDK (Software Development Kit)
  • ADT (Android Development Tools)
  • Robotium
  • XING Android App

After installing all the components a new Android Test Project can be created within Eclipse (see figure 1)

Figure 1: Android test project

The next step is to adapt the AndroidManifest.xml file (see code listing 1). In this file, the XING Android app package name has to be entered after which Robotium is able to communicate with the app.

Code listing 1: The AndroidManifest xml file

<!--?xml version="1.0" encoding="utf-8"?-->

To start the first Robotium test class, a Java class that extends the Robotium Framework class ActivityInstrumentationTestCase2 has to be created. This class then provides methods and activities to interact with the app. The core of the automated tests is the Robotium object Solo which provides access to the entire Robotium framework along with all of the provided methods. The first step before using the object is to initialize it in the setUp() method (see code listing 2). In this method the object is initialized with central Android activity.

Code listing 2: The setUp() method

@Override
protected void setUp() throws Exception {
        solo = new Solo(getInstrumentation(), getActivity());
}

The really simple example in code listing 3 aims to demonstrate the functionality of Robotium and the solo object. The example shows a test involving the login process. The testLogin() method is created to test the login process. Within this method, the following lines of Java code were written:

Code listing 3: Simple test method for the login process

public void testLogin()throws Exception{
	solo.enterText(0, "Testusername");
	solo.enterText(1, "secret");
	solo.clickOnButton("Secure Login");
	solo.waitForActivity("com.xing.android.activities.SpinnerLoginActivity", 3000);
	solo.assertCurrentActivity("Assertion failed, wrong Activity", "DashboardActivity");      
	assertTrue(solo.searchText("News"));
}

The solo object provides access to the enterText method which requires two parameters that allow some text to be entered into a text input field within the app. The first value of the parameters represents the IDs of the input text field on the login screen. The second parameter is the string that should be entered. The clickOnButton() method “clicks” the button to login the user (see figure 2).
The waitForActivity()method waits for the login activity of the app until the user is logged in. After a successful login, Robotium uses the assertCurrentActivity() method to verify if the dashboard activity is shown (see figure 3). At the end of the test method, a JUnit assertTrue verification is performed to check whether or not the dashboard and /or “News” button is/are visible.

Figure 2: Login
Figure 3: Dashboard

At the end of a test run, the tearDown() method is called to close all activities and to clean up the solo object (code listing 4).

Code listing 4: The TearDown() method

@Override
public  void tearDown() throws Exception {
 	try {
 		solo.finalize();
 
 	} catch (Throwable e) {
		e.printStackTrace();
	}
	getActivity().finish();
 	super.tearDown();
}

One the automated tests have been developed they can then be run on Android devices. To execute them, start the Robotium test project as Android JUnit test. During the test run, JUnit generates a report, displays error messages if any problems occur (see figure 4).

Figure 4: JUnit test run report

Robotium offers a number of benefits in terms of creating an automated regression test suite for Android devices. Developing stable and powerful automated tests is really easy and can save a lot of time by following a few programming rules. One of the biggest advantages of Robotium is that automated tests can be executed on real devices.

Until now, the core functions of the XING Android app, i.e. the login process, messages, news, visitors, personal profiles and search features, were automated using Robotium. Automated test development is an ongoing process during which every new feature of the app will be automated. Tests will be executed every day to give the Android team the guarantee that existing functions are still working, even after a code change. Changes that lead to errors are now found much earlier in the development process by our agile team, which in turn means faster and more effective development work.

The latest XING Android App is available for download here https://market.android.com/details?id=com.xing.android

About the author

Daniel KnottDaniel Knott works as a Manager Quality Assurance in XING’s mobile team. He likes to automate test cases using technologies such as Robotium, KIF(Keep It Functional), Selenium and Java.
XING Profile »


71 thoughts on “Robotium @ XING. Automated regression tests on mobile Android devices

  1. Hi Daniel,
    I am testing an application which first requires the user to log in. Is there a way to first login and then in all other tests continue from that point onward? I know I can call Login() in every test, but that seems redundant.

    Maybe there is a way to save the appication state and just restore it at the beginning of each test?

    Thank you for your time,
    Dino

  2. @Dino, write a HelperClass where you implement the login and call this method in each testmethod. i think this is the best way to do.

    @Shah, no there is no way to use TestNG.

    Greets,

    Daniel

  3. Hi,

    I’m automating for our launcher. I’m facing problem with scroll/swap between workspace. Code which i created is working fine for 2.3. but for 4.0.showing nullpointerexception. here is my exception:

    Error in testSwitchWorkspace:
    java.lang.NullPointerException
    at com.avatar.icslauncher.test.workspaceTest.testSwitchWorkspace(workspaceTest.java:76)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
    at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
    at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
    at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
    at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
    at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:545)
    at com.ninetology.android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:45)
    at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1551)
    .

    If need any details feel free to ask. Thank you.

  4. regarding my previous workspace swap question, i forgot to mention that i’m using Motionevent.obtain(…….) to simulated thick finger touch and move across screen.

  5. Hi Daniel

    I automated my app using Robotium. Now I want to create a framework around it. But I am quite new to all this automation and framework thing. Can you please help me out regarding the same?

  6. Hi Varun,

    how can I help you!? What do you want to do with the framework? How should it look like? Please provide some more information.

    Greets,

    Daniel

  7. Hi Daniel,

    I am interested in the robotium in automation testing.
    Currently I meet an issue is that how could I launch application like “Office Suite” in market. It seems the application launched in different process to our test apk. Another question is that how could we test the appliciaon like “Angrybird” game. It seems after launch, there is no any element we can control.
    Any info form you is appreciated!

    Thanks!
    Geo Zhang

  8. Hi Daniel,

    Thank you for your feedback.
    “Office Suite” is an applicaiton in Google market.
    Both “Office suite” and “Angrybird” has been resigned and keeps running in device. Now, I can launch Angrybird, but after launch, I can not get any element for the application. For “Office Suite”, I can not launch it at all.
    I am not sure if my question is clear now.

    Thanks!
    Geo Zhang

  9. Hi Geo,

    mh ok. Do you have any kind of log message? Maybe there is a simple reason why you are not able to start office suite!
    That is really strange why you can’t access any elements. Have you tried to click on text using the method: solo.clickOnText(“Text”);? Or have you tried with a simpler app like a text editor or something!?

    Greets

  10. Hi Daniel,

    I would like to know if it is possible to log in to Facebook or twitter ( any web based application ) using Robotium.
    I tried doing this but was not successful, the log in fields could never be entered in a web based view.

    Thanks,
    Darsh

  11. Hi Darshan,

    Robotium is not able to interact with web views! You can just automate native UI elements with Robotium. Try Calabash! This tool is able to interact also with web views.

    Greets,

    Daniel

  12. Hi Daniel,
    how can we get screen shot of application flow using robotium and is there is way to generate rich html report in robotium

  13. Hi Rohit,

    you can use the method solo.takeScreenshot() or solo.takeScreenshot(name) to take screenshots of your application under test. The screenshot will be saved on the device sd card (/sdcard/Robotium…). After the testrun you need to pull the screenshots with adb pull from the device to your system. Generating a report is possible by using the JUnit results.

    Greets,

    Daniel

  14. public void goToGoogle() {
    FirefoxProfile profile = new FirefoxProfile();
    WebDriver driver = new FirefoxDriver();
    driver.manage().timeouts().implicitlyWait(3, TimeUnit.SECONDS);
    driver.get(“http://www.google.com”);
    driver.quit();
    }

    I called this function in my robotium test class, while i am running test as android junit test.. i am getting the error like this.

    java.lang.NoClassDefFoundError: org.openqa.selenium.firefox.FirefoxProfile
    at com.android.pages.Rungoogle.goToGoogle(Rungoogle.java:11)
    at com.android.test.listing.TestMyAdsPage.testVerifyMessageInMyAdsPage(TestMyAdsPage.java:54)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:204)
    at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:194)
    at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:186)
    at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)
    at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)
    at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:529)

    Actually what i need is i want to open firefox system itself .. same time i want to run my script in mobile, in between i want to open firefox in my system..

    Please help me to resolve the issue.

  15. Hello Daniel,

    I am new to Robotium. I want to test some apps using this tool.

    I want to display a home screen with button after the display of splash screen for 3 seconds.
    i.e. when an app is opened, splash screen should display for 3 seconds and then home page(another activity) should be displayed.
    Can you say, how can I write the testsuite for the same.

    Thanks in advance…

  16. Hello,

    Thank you Daniel for your links. I have already gone through the mentioned links. In the sample project mentioned, it was dealt with only a single activity. My concern is to test the project which is having multiple activities. Anyhow, my last issue has been fixed.

    And I need one clarification regarding the below statement which a saw in a page.

    “the test project is locked to the target Package. Going outside of that target package is not allowed by the Android platform. Therefore you will need 2 test projects, one for each application.”

    Please explain the above in detail if possible.

    And if a single project is having multiple packages, is it possible to create the test cases for it, without creating two test projects. I need clear idea on this concept.

    Awaiting for your reply….

  17. Hi,

    I am very much new for this .I have successfully run my test cases but can you pls guide me to prepare report in .html or .xml format.

    Thanks,
    Ranjana

  18. Hi!! I am trying to automate mobile application but I am unable to click on Action Bar that is Sherlock Action Bar.Please tell the solution to click on menus in Sherlock Action Bar.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>