Getting Started

Calabash

Getting Calabash for Android

https://github.com/calabash/calabash-android

General Installation Steps

The general installation guide for Calabash for Android is found here https://github.com/calabash/calabash-android/blob/master/documentation/installation.md. Once all the needed tools are installed (Ruby and Android SDK), then:

$ sudo gem install calabash-android

and command calabash-android should be available in path.

Creating a New Test

$ mkdir new-calabash-test
$ cd new-calabash-test
$ calabash-android gen

This generates a new features-directory inside new-calabash-test directory. Features directory contains my_first.feature-file, step_definitions-directory and support-directory. For a simple test only edit my_first.feature-file. Here is the list of predefined steps required: https://github.com/calabash/calabash-android/blob/master/ruby-gem/lib/calabash-android/canned_steps.md

Running The First Test

Here is an example Calabash test:

Feature: Click items
  Scenario: Run whole app
   When I press view with id "radio0"
   Then I wait
   Then I press view with id "radio1"
   Then I wait
   Then I press view with id "radio2"
   Then I take a screenshot
   Then I wait
   Then I enter text "Hello Calabash" into field with id "editText1"
   Then I take a screenshot
   Then I wait
   Then I press view with id "button1"
   Then I wait
   Then I take a screenshot
   Then I go back
   Then I wait
   Then I press view with id "radio1"
   Then I wait
   Then I press view with id "button1"
   Then I wait
   Then I take a screenshot

NOTE: Calabash requires the application to have INTERNET permission enabled in the AndroidManifest.xml

If the application is not yet signed you can do it with Calabash:

$ calabash-android resign {path-to-apk}

Now you should be ready to run Calabash test locally. Go to new-calabash-test-directory and run:

$ calabash-android run {path-to-apk}

Output should look something like this:

Feature: Click items
 Scenario: Run whole app # features/my_first.feature:3
  2485 KB/s (554161 bytes in 0.217s)
  2282 KB/s (100963 bytes in 0.043s)
  When I press view with id "radio0" # calabash-android-0.4.14/lib/calabash-android/steps/press_button_steps.rb:13
  Then I wait # calabash-android-0.4.14/lib/calabash-android/steps/progress_steps.rb:5
  Then I press view with id "radio1" # calabash-android-0.4.14/lib/calabash-android/steps/press_button_steps.rb:13
  Then I wait # calabash-android-0.4.14/lib/calabash-android/steps/progress_steps.rb:5
  Then I press view with id "radio2" # calabash-android-0.4.14/lib/calabash-android/steps/press_button_steps.rb:13
  Then I take a screenshot # calabash-android-0.4.14/lib/calabash-android/steps/screenshot_steps.rb:9
  Then I wait # calabash-android-0.4.14/lib/calabash-android/steps/progress_steps.rb:5
  Then I enter text "Hello Calabash" into field with id "editText1" # calabash-android-0.4.14/lib/calabash-android/steps/enter_text_steps.rb:25
  Then I take a screenshot # calabash-android-0.4.14/lib/calabash-android/steps/screenshot_steps.rb:9
  Then I wait # calabash-android-0.4.14/lib/calabash-android/steps/progress_steps.rb:5
  Then I press view with id "button1" # calabash-android-0.4.14/lib/calabash-android/steps/press_button_steps.rb:13
  Then I wait # calabash-android-0.4.14/lib/calabash-android/steps/progress_steps.rb:5
  Then I take a screenshot # calabash-android-0.4.14/lib/calabash-android/steps/screenshot_steps.rb:9
  Then I go back # calabash-android-0.4.14/lib/calabash-android/steps/navigation_steps.rb:1
  Then I wait # calabash-android-0.4.14/lib/calabash-android/steps/progress_steps.rb:5
  Then I press view with id "radio1" # calabash-android-0.4.14/lib/calabash-android/steps/press_button_steps.rb:13
  Then I wait # calabash-android-0.4.14/lib/calabash-android/steps/progress_steps.rb:5
  Then I press view with id "button1" # calabash-android-0.4.14/lib/calabash-android/steps/press_button_steps.rb:13
  Then I wait # calabash-android-0.4.14/lib/calabash-android/steps/progress_steps.rb:5
  Then I take a screenshot # calabash-android-0.4.14/lib/calabash-android/steps/screenshot_steps.rb:9
  1 scenario (1 passed)
  20 steps (20 passed)
  0m47.002s

If your test takes screenshots they should be as png-files in new-calabash-test-directory.

Run Calabash Android Test in Bitbar Testing

Access Bitbar Testing and login. The following steps will help you get your Calabash tests up and running on over 250 Android devices:

  1. Login to Bitbar Testing

  2. Create a new project

    create new project

  3. Upload your application (.APK)

    upload apk

  4. Create zip-package from your features-directory

  5. Upload test zip-package
    upload zip
  6. Select devices
    select devices
  7. Check advanced configurations

  8. Run the project

  9. Verify results

Advanced Settings

When creating a Calabash test project, it is possible to defined some environment variables to be used during test execution.

In Bitbar Testing it is possible to give values to the two pre-existing environment variables CALABASH_TAGS and CALABASH_PROFILE, in the advanced settings of a test run. These can be used to better orchestrate in which order tests get executed.

image5

Users of Private and Enterprise Bitbar Testing clouds can create their own additional variables.

Typical Questions

Packaging Your Tests

The basic test.zip file must contain folder called features. You can include other files too like Gemfile or config folder with cucumber.yml file.

How to Use Tags in Tests

Add a configuration folder with cucumber.yml to your zip file. This example will only run tests marked with tag smoke.

# config/cucumber.yml
##YAML Template
---
default: >
--tags @smoke

Currently only default profile is supported. If you have other profiles locally you need to edit cucumber.yml to use default profile before testing in Bitbar Testing.

RESET_BETWEEN_SCENARIOS Is Not Working

Calabash Android does not support this option for real devices.