Running tests

Listing available frameworks

curl -u <api-key>: https://cloud.bitbar.com/api/me/available-frameworks

Starting test run

These are put into a configuration file like below.

{
  "osType":"ANDROID","projectId":PROJECT_ID,
  "files":[
      {"id":APP_FILE_ID},
      {"id":TEST_FILE_ID},
      {"id":DATA_FILE_ID}
  ],
  "frameworkId":FRAMEWORK_ID,"deviceGroupId":DEVICE_GROUP_ID
}
curl -H 'Content-Type: application/json' -u <api-key>: https://cloud.bitbar.com/api/me/runs --data-binary @configuration

Note: Mandatory fields are: osType, frameworkId, files (depending on the framework), deviceGroupId or deviceIds

Details about the configuration fields:

  1. applicationPassword - only for AppCrawler runs, if application has login screen
  2. applicationUsername - same as above
  3. deviceGroupId - specify device group devices for the run
  4. deviceIds - specify device ids for the run (higher priority than deviceGroupId)
  5. deviceLanguageCode - specify device language (US by default)
  6. files - files configuration. More details below
  7. frameworkId - specify framework for the run
  8. hookURL - url to be POSTed when run is finished (must be public)
  9. instrumentationRunner - for Android Instrumentation framework. Default: android.test.InstrumentationTestRunner
  10. launchApp - for UI Automator framework. Define if the app should be launched. (Can be launched by the test)
  11. limitationType - for Android Instrumentation - related to limitationValue. Values: PACKAGE, CLASS
  12. limitationValue - for Android Instrumentation - related to limitationType. Define what test cases should be launched
  13. maxAutoRetriesCount - define how many times test can be retried (in case of failures)
  14. osType - define osType of the used devices: ANDROID, IOS, DESKTOP etc.
  15. projectId - define project in which run should be created. When empty - a new project is created
  16. projectName - name of the project to be created when projectId is not defined. Generated when empty
  17. scheduler - define scheduler of the run:
  • PARALLEL - all device sessions can be executed simultaneously.
  • SERIAL - all device sessions start one by one (single at the time)
  • SINGLE - start only on first available device (rest of them are excluded)
  • ALL_INSTANCES - start on all instances from all selected device models (for Administrators)
  1. screenshotDir - directory of screenshots on devices. Default value: /sdcard/test-screenshots
  2. testRunId - when set new device sessions are attached to existing test run
  3. testRunName - name of the run
  4. testRunParameters - list of additional parameters (key, value)
  5. timeout - timeout of the run after which run is interrupted and user can expect partial results
  6. uiAutomatorTestClasses - for UI Automator framework
  7. videoRecordingEnabled - define if video should be recorded. Enabled by default
  8. withAnnotation - for Android Instrumentation framework. Run test with specified annotations
  9. withoutAnnotation - for Android Instrumentation framework. Run test without specified annotations

The configuration can change in the future. The most up to date structure can be found in open-sourced repository

Details about files configuration:

  1. id - id of the file
  2. action - optional field (Cloud tries to recognize file types and decide of its own). It can be used if the process fails or you want to be more specific
  • COPY_TO_DEVICE - copy file to device sdcard
  • INSTALL - install file on the device
  • RUN_TEST - run file as a test