Cloud Side Appium

Cloud side Appium testing is when you upload your existing Appium tests to Bitbar Cloud and then either using the API or the cloud UI select and start test execution on multiple devices. The benefits of this approach are, you do not need to modify your Appium tests, execution of tests can be parallelized on multiple devices and you are informed when the execution is finished.

The below used sample code can be downloaded from our git repository. Here we use Python, you can also find tests suites in other supported languages like Java, ruby, C#.

Running Sample in Cloud

Packaging The Tests

All needed files for testing need to be uploaded to cloud as zip package. This package must contain test files, data files and needs to have a Shell script to launch test execution at the root level of the package. So there are a few easy steps to be taken before you can upload your current tests to the cloud for Server Side execution. Find them below.

Note, that this is different from Client Side execution, read more about it under the Client Side chapter.
  1. Have your app ready
  2. Important files
  3. Create your test suite Zip file
  4. Upload and execute

1. Have your app ready

One important file is of course your application. This needs to be uploaded alongside the test suite zip. Make sure it is an application made for testing. If it is an .ipa, you can read about the packaging.

2. Important files

For the cloud side execution to work properly, it is important to have the test suite zip outfitted with the right files. Here we will go through the files and also the correct makeup of the test zip. The most important is the run-tests.sh file, it is responsible for starting the test execution in the cloud and also installing requirements. Depending whether you are testing iOS or Android, you’ll need to rename run-tests_android.sh or run-tests_ios.sh to run-tests.sh depending which environment you want to test.

In the run-tests.sh file make sure that the ${TEST} variable points to correct file name. This is the name of the main Python test script file. Other modifications are not needed. In this python example we use the Bitbar Sample Test script so our TEST variable is the following:

TEST=${TEST:="BitbarSampleAppTest.py"} #Name of the test file

For the sake of this example, let’s decide we want to run an Android test, so we need to rename run-tests_android.sh to run-tests.sh.

$ cp run-tests_android.sh run-tests.sh

Example zip makeup (Python):

run-tests.sh:            This file is responsible for starting the test execution in the cloud
BitbarSampleAppTest.py: The actual test files written in Python.
requirements.txt:       Lists the required Python packages that need to be installed
                        for the test to be executable, eg. AppiumPythonClient or RobotFramework.

Python sample files can be found Here.

Example zip makeup (Java):

run-tests.sh:            This file is responsible for starting the test execution in the cloud
Pom.xml:                Maven configuration file
src/ folder:            Folder containing tests and other test files,
                        note that the test files themselves are not at root level.

Java sample files can be found Here. The README has a more detailed explanation about the repository structure and how to use it.

3. Create your test suite Zip file

The above python sample has the following content:

$ ls -lrt
-rwxr-xr-x  1 user  staff    60 Feb 28 16:31 requirements.txt
-rw-r--r--  1 user  staff  4667 Mar  9 15:38 TestdroidAppiumTest.py
-rw-r--r--  1 user  staff  2687 Mar 13 11:04 BitbarSampleAppTest.py
-rwxr-xr-x  1 user  staff  1263 Mar 14 16:10 run-tests_ios.sh
-rwxr-xr-x  1 user  staff  1382 Mar 14 16:11 run-tests_android.sh

Now creating the test zip (called here test.zip) for upload is done as:

$ zip -r tests.zip *
  adding: BitbarSampleAppTest.py (deflated 67%)
  adding: TestdroidAppiumTest.py (deflated 70%)
  adding: requirements.txt (deflated 2%)
  adding: run-tests.sh (deflated 47%)
  adding: run-tests_android.sh (deflated 47%)
  adding: run-tests_ios.sh (deflated 44%)

Note, the run-tests.sh needs to always be at the root of the package and it is copied and renamed from run-tests_android.sh.

4. Upload and Execute

Now it is time to use your newly created files and start a test run. Navigate to cloud.bitbar.com and login with your credentials.

  • Start by selecting Test Run Creator or create a New Project in the projects view.
  • Create a new test run for this project
  • Upload tested application (apk/ipa) in the “Application” step
  • Upload the above created zip using the “Upload test file” step
  • Choose the device group to use or create a new group for this run
  • Select optional advanced options
  • Start your test run!

Once the test execution has finished, test results become available in the cloud.

Additional Considerations

To get all your test artifacts presented in Bitbar Cloud you should take the following additional steps into consideration.

  • To get screenshots and test results visible in the cloud, make sure your test scripts store screenshots in a “screenshots” directory at the root of the test files directory.
  • Also if your tests create JUnit result files, name them as TEST-all.xml and locate it at the root of the test files directory

After a test run of this sample test the output should look like so.

$ ls -lrt
-rw-rw-r-- 1 user staff    4667 Jun  8 01:05 TestdroidAppiumTest.py
-rwxrwxr-x 1 user staff    1263 Jun  8 01:05 run-tests_ios.sh
-rwxrwxr-x 1 user staff    1382 Jun  8 01:05 run-tests_android.sh
-rwxr-xr-x 1 user staff      60 Jun  8 01:05 requirements.txt
-rw-rw-r-- 1 user staff    2687 Jun  8 01:05 BitbarSampleAppTest.py
-rwxrwxr-x 1 user staff    1382 Jun  8 01:06 run-tests.sh
-rw-rw-r-- 1 user staff 1595408 Jun  8 01:09 get-pip.py
-rw-rw-r-- 1 user staff    5572 Jun  8 01:09 TestdroidAppiumTest.pyc
drwxrwxr-x 2 user staff    4096 Jun  8 01:09 screenshots
-rw-rw-r-- 1 user staff    3129 Jun  8 01:09 TEST-all.xml
drwxrwxr-x 2 user staff    4096 Jun  8 01:09 test-reports

Running Cloud Sample Locally

As the changes needed to make local tests run in the Bitbar Cloud are minor. Therefore it is possible to execute the cloud test zip locally as well.

This example works for both Android and iOS devices. Depending which you want to test, rename run-tests_android.sh or run-tests_ios.sh to run-tests.sh. To get started you need to have curl, unzip, appium and adb for Android or Xcode for iOS installed on your test machine. Rest of the required dependencies are listed in the requirements.txt file of this sample directory. The run-tests.sh takes care of installing the remaining tools and libraries. To read more about how to set up the Environment, check out these Instructions.

Changing Settings

  • In run-tests.sh remove or comment out, the zip extraction, requirements installations and starting of the Appium server as these are ment for the cloud only or parametrize to fit your local environment needs. The cloud dependencies can be found between the lines:

    ##### Cloud testrun dependencies start

    ##### Cloud testrun dependencies end.

  • Launch Appium server as normally when running local Appium tests. Appium for iOS needs device ID in launch parameters

  • run-tests.sh assumes the tested apk/ipa is named as application.apk or application.ipa and is located at the root of the working directory. Alternatively update run-tests.sh to use some other path.

  • Launch tests by running the run-tests.sh script.

Tips and Tricks

Use the samples as a base for your test suite, this way you already have the correct folder makeup, replace or modify test files according to your needs.