Desktop Browsers

Running desktop browser tests on Bitbar cloud makes use of the Bitbar build API and overall behavior is similar to eg. Appium server side test runs currently on the testing cloud.

The samples below use the jq tool for prettifying the JSON response messages. This is not required to get tests working.

Creating a Run

  1. Creating the test package. The Selenium test suite needs to be packaged into a zip file that has at its root a run-tests.sh (for Linux and Mac based browser test runs) or a PowerShell run-tests.ps1 file for Windows based test runs. This script is what is called by the Bitbar test environment to start the test execution. In addition to starting the test run, the script can be used to install additional packages from the Internet, make changes to the environment or set environment variables to control the test execution.

    Archive:  linux-tests.zip
    Zip file size: 2066 bytes, number of entries: 3
    -rw-r--r--  3.0 unx       48 tx stor 18-Aug-29 09:51 requirements.txt
    -rw-r--r--  3.0 unx     2663 tx defN 18-Sep-04 14:17 MyTest.py
    -rwxr-xr-x  3.0 unx     1505 tx defN 18-Sep-07 16:50 run-tests.sh
    3 files, 4216 bytes uncompressed, 1576 bytes compressed:  62.6%
    

    In this example the run-tests.sh needs to contain the call to python MyTest.py to start the execution of the test suite.

  2. Upload Test Package to Cloud and store the upload file’s response ID for starting the build.

    curl -X POST -u "${APIKEY}:" ${URL}api/v2/me/files -F "file=@linux-test.zip" | jq '.'
    {
      "id": 198192,
      "selfURI": null,
      "createTime": 1535443943000,
      "fileProperties": [],
      "state": "READY",
      "name": "linux-test.zip",
      ...
    }
    
  3. All test runs need to be associated with a build job, that needs to be created. You’ll want to have one for each project that you need to run desktop browser tests. Create new build as such, replace BUILD NAME with the name you want to give your project.

    curl -X POST -u "${APIKEY}:" ${URL}cloud/api/v2/me/jobs?type=BROWSER_TESTING&name=<BUILD NAME> | jq '.'
    
  4. Your cloud will have a number of runners (browser and operating system configurations) that you get from your cloud administrator or can query over the API.

    With this information you should create your Jenkins build configuration. In minimal, the configuration needs to have the test package file ID and the executor ID. These are put into a start-build file like below.

    {
      "fileId":198192,
      "executorId":15,
      "resultsConfig":[],
      "configuration":null
    }
    
  5. Now everything is ready for starting your test.

    curl -X POST -H 'Content-Type: application/json' -u "${APIKEY}:" ${URL}api/v2/me/jobs/${JOB_ID}/builds --data-binary @start-build
    
  6. Get results link from a finished job. Results can be downloaded following the directUrl link in response message.

    curl -s -X GET -u "${APIKEY}:" ${URL}api/v2/me/jobs/${JOB_ID}/builds/${BUILD_ID}/output-file-set/files | jq '.'
    

Aborting Runs

It is possible some test jobs get hanging and don’t release the executors (browser + operating system runners). These jobs need to be terminated manually or add some cleanup job to your Jenkins.

Listing All Builds

Using the below curl command, you can list all your jobs and find any job that is not in the FINISHED state.

curl -s -X GET -u "${APIKEY}" ${URL}/api/v2/me/builds?sort=id_d | jq '.'
Aborting a Build

To release an executor from a stuck build let’s call an abort on it, using the build and job IDs (from previous query).

curl -s -X GET -u "${APIKEY}" ${URL}/cloud/api/v2/jobs/${JOB_ID}/builds/${BUILD_ID}/abort | jq '.'