Client Side Execution

Overview

With Bitbar Testing it is possible to execute Appium tests similarly as when running the tests on your local machine. The difference is that Appium server and the tested device are in Bitbar Testing Cloud.

The below steps guide you how to get your existing Appium tests running against devices hosted in cloud.

Upload Your App To Cloud

First the app under test needs to be uploaded to Bitbar Testing. This can be done either using curl on the command line as below or some automated CI solution.

The prefered way to identify yourself to Bitbar Testing is using the apiKey, a unique token that can be copied or changed at any time from user’s My account in Bitbar Testing cloud.

$ curl -H "Accept: application/json" -u xYY5...PeOA6: -F myAppFile=@"/absolute/file/path/example.apk" http://appium.testdroid.com/upload
{"status":0,"sessionId":"bb8d4336-e6ea-4ba7-9b4c-a6824f1c60aa","value":{"message":"uploads successful","uploadCount":1,"rejectCount":0,"expiresIn":1800,"uploads":{"myAppFile":"bb8e4336-e6ea-4ba7-9b4c-a6824f1c60aa/Testdroid.apk"},"rejects":{}}}

Above response is a JSON message where the name of the uploaded app in cloud is the value of “myAppFile”:”bb8e4336-e6ea-4ba7-9b4x-a6824f1c60aa/Testdroid.apk”. The value “bb8e4336-…” is needed later when defining Appium capabilities.

Edit Desired Capabilities

Common values used in tests:

  • screenshot_dir - where should screenshots be stored on your local drive

  • testdroid_apiKey - a personal unique key allowing you to connect to Bitbar Testing without using username and passwords in tests. Api key is found under “My account” in Bitbar Testing Cloud UI.

  • testdroid_project - the project name in Bitbar Testing. Each project must have a unique name, which can be modified (in Cloud)

  • testdroid_testrun - name of this test run inside of testdroid_project. Each test run can have it’s own name (eg. date + time)

  • testdroid_app - name of the app uploaded using upload.py script. Example filename could be ‘f4660af0-10f4-46e9-9345-0633f497b0d2/Testdroid.apk’. If the app has been uploaded through the UI then the value “latest” can be used too.

In addition to the above capabilities, default Appium capabilities are supported. It is worth checking the list of supported Desired Capabilities.

Selecting a Device

On client side Appium runs Bitbar Testing clouds gives preference on providing some available device over an exact match. This preference can be changed with the desired capability testdroid_findDevice. This capability allows the user to get a similar device to the one requested or the exact requested. Device similarity is defined by device name similarity matching. This means running the same test twice with testdroid_findDevice='true', the device selected by cloud can be different.

To enforce selecting a specific device, one should check for the exact name from cloud and set the testdroid_findDevice capability to false. If the selected device is currently busy, a new session is tried for a few minutes. The connection is closed if device does not become available during this time.

Updating Existing Tests For Cloud

Updating Capabilities

Online samples on Github, accessible here, help getting started on running Appium tests. The samples are available in Python, Java, C# and Ruby.

For running your existing Appium runs against Bitbar cloud, your existing desired capabilities need to be updated. Mainly authentication information and selecting the appropriate device need to be provided. The app name is the name from the above curl upload. Below example shows the additional capabilities that need to be added.

Before

'app' : 'com.bitbar.testdroid.BitbarIOSSample',
'device' : 'iphone'

After changes

'app' : 'com.bitbar.testdroid.BitbarIOSSample',
'device' : 'iphone',
'testdroid_apiKey' : '<your apiKey>',
'testdroid_project' : 'Appium iOS Project',
'testdroid_testrun' : 'Test run 1',
'testdroid_device' : 'iPhone 5c',
'testdroid_app' : 'bb8e4336-e6ea-4ba7-9b4x-a6824f1c60aa/Testdroid.ipa',
'testdroid_locale' : 'fi_FI' # optional 

Change WebDriver Address

When running Appium locally, the web driver address is running on localhost. When using the cloud the wed driver is located at https://appium.testdroid.com/wd/hub.

Upload Results to Cloud (Optional)

If your test suite generates a JUnit XML results file, you can upload the XML to Bitbar Cloud. Doing this will allow you to check your test cases and their run statuses on the web UI, and let you download test reports in various formats.

  1. Add the testdroid_junitWaitTime Desired Capability in your TestScript with an approoriate value in seconds. Eg. 60 seconds should be sufficient.

  2. Get Appium session ID from your test script (after the WebDriver connection has been established) by printing the property:

self.utils.log("Driver session id: "+ self.driver.session_id)

Session ID printed to command line

  1. After your test run has finished, and JUnit XML has been generated, use Curl to upload the XML to Cloud:
curl -s -F result=@"/absolute/file/path/TestOutput.xml" https://appium.testdroid.com/upload/result?sessionId=<sessionId>

Link to Java Sample

Our Github has a Java specific sample for those using Java.