Flutter Support

This guide shows how to run Flutter integration tests for Android real device and iOS simulator (real iOS devices are not supported but can be tried to run tests on) on Bitbar Cloud. Unit and widget tests can be run in the same way, but tests don’t run on actual device, cloud device time is still used but the device is just idling while tests run.

Android

Android tests are run on Linux OS. Flutter (check current version with “flutter doctor” command) and Android SDK version 29.0.2 are preinstalled in VM machine.

Create a shell script file called run-tests.sh. This file will contain commands to execute the test.

  1. Unzip test files:
unzip tests.zip
  1. Accept SDK licences:
yes | flutter doctor --android-licenses
  1. Run Flutter doctor:
flutter doctor
  1. (optional) go to flutter project directory (sample command):
cd my_app
  1. (optional) clean project folder (build / packages)

flutter clean

rm -r .packages

rm pubspec.lock

  1. Run Flutter integration tests (sample command):
flutter drive --target=test_driver/main.dart
  1. (optional) if test results (need to be a file called TEST-all.xml) or screenshots should be displayed in cloud UI:
  • Go back to root directory (test result files are searched from root directory):

    cd ..

  • Move “TEST-all.xml” to root directory:

    mv my_app/TEST-all.xml TEST-all.xml

  • Create “screenshots” directory and move screenshots there (sample command):

    mkdir -p screenshots

    mv /tmp/screenshots/test/ screenshots

Create a zip file containing run-tests.sh file and your Flutter project directory. Note that “run-tests.sh” file must be in the root of the zip file, otherwise it is not found. Sample command how to create zip: zip -r android-test-files run-tests.sh my_app. Upload this file to cloud and use it as the “Use to run the test” file.

Appium Server Side type test project expects an APK file to be uploaded. Upload a small dummy app file and use it as “Install on the device”. Note that app need to be able to install to device, otherwise test will fail. Actual Flutter app to test is built during test run.

How to create a Flutter Android test run in Bitbar Cloud

  1. Create Appium Android Server Side type test run.

    image0

    image1

  2. Upload test zip file and actual app APK or small dummy app APK file (app need to be able to install to device), test does not use this app.

    image2

  3. Select devices

  4. Make sure to set long enough test time-out period

  5. Run tests

IOS (simulator)

IOS simulator test runs spend actual real device test time, real iOS device is just idling while simulator is running tests. IOS tests are run on Mac OS. Flutter is preinstalled in VM machines. Cocoapods need to be installed, it can be done using “run-tests.sh” file.

Create a shell script file called run-tests.sh. This file will contain commands to execute the test.

  1. Unzip test files:
unzip tests.zip
  1. Install cocoapods:

sudo gem install cocoapods

pod setup

  1. Run Flutter doctor:
flutter doctor
  1. (optional) go to flutter project directory (sample command):
cd my_app
  1. (optional) clean project folder (build / packages)

flutter clean

rm -r .packages

rm pubspec.lock

  1. Install iOS simulator:
npm install -g ios-sim
  1. Start simulator:
ios-sim start --devicetypeid "iPhone-8"
  1. Find simulator device from Flutter devices:
flutter devices > devices.txt
cat devices.txt | while read line
do
case "$line" in
*simulator*)
  TEST_DEVICE=$line
  RUN_DEVICE=`echo ${TEST_DEVICE} | awk '{print $1, $2}'`
  echo $RUN_DEVICE > run_device.txt
  echo "device"
  cat run_device.txt
  ;;
  *)
  ;;
esac
done
  1. Run Flutter integration tests using the simulator (sample command):
flutter drive -d "`cat run_device.txt`" --target=test_driver/main.dart
  1. (optional) if test results (need to be a file called TEST-all.xml) or screenshots should be displayed in cloud UI:
  • Go back to root directory (test result files are searched from root directory):

    cd ..

  • Move “TEST-all.xml” to root directory:

    mv my_app/TEST-all.xml TEST-all.xml

  • Create “screenshots” directory and move screenshots there (sample command):

mkdir -p screenshots

mv /tmp/screenshots/test/ screenshots

Create a zip file containing run-tests.sh file and your Flutter project directory. Note that “run-tests.sh” file must be in the root of the zip file, otherwise it is not found. Sample command how to create zip: zip -r ios-test-files run-tests.sh my_app. Upload this file to cloud and use it as the “Use to run the test” file.

How to create a Flutter iOS simulator test run in Bitbar Cloud

  1. Create Appium iOS Server Side type test run.

    image3

    image4

  2. Upload test zip file and actual app ipa or small dummy app ipa file (need to be able to install to actual real iOS device, no simulator builds), test does not use this app.

    image5

  3. Select devices (actual selected device is not used to run test on)

  4. Make sure to set long enough test time-out period

  5. Run tests

IOS real device

IOS real device testing is not guaranteed to work because of iOS app signing (no proper documentation from Flutter yet available).

IOS Flutter app need to be built locally and then the Runner app (converted to ipa) will need to be uploaded to cloud (as “Install on the device”). The Runner ipa is then re-signed in cloud. During the test run the app will need to be moved to build/ios/iphoneos/ directory. Test is then ran using –no-build -flag. Currently 32 bit iOS devices and iOS 13 devices don’t work with Flutter on Bitbar Cloud. Read more about how to create .ipa from .app.

Create a shell script file called run-tests.sh. This file will contain commands to execute the test.

  1. Unzip test files:
unzip tests.zip
  1. Install cocoapods:

sudo gem install cocoapods

pod setup

  1. Run Flutter doctor:
flutter doctor
  1. Unzip re-signed Flutter iOS app (it will be called “application.ipa”):

mv application.ipa application.zip

unzip application.zip

  1. Get rid of “Payload” directory:
mv Payload/Runner.app Runner.app
  1. Move Runner app to build directory (create build directory if it is not already in project directory)(sample command):
mv Runner.app my_app/build/ios/iphoneos/Runner.app
  1. (optional) go to flutter project directory (sample command):
cd my_app
  1. Run Flutter integration tests without building the app (sample command):
flutter drive -v --no-build --target=test_driver/main.dart
  1. (optional) if test results (need to be a file called TEST-all.xml) or screenshots should be displayed in cloud UI:
  • Go back to root directory (test result files are searched from root directory):

    cd ..

  • Move “TEST-all.xml” to root directory:

    mv my_app/TEST-all.xml TEST-all.xml

  • Create “screenshots” directory and move screenshots there (sample command):

mkdir -p screenshots

mv /tmp/screenshots/test/ screenshots

Create a zip file containing run-tests.sh file and your Flutter project directory. Note that “run-tests.sh” file must be in the root of the zip file, otherwise it is not found. Sample command how to create zip: zip -r ios-test-files run-tests.sh my_app. Upload this file to cloud and use it as the “Use to run the test” file.

How to create a Flutter iOS real device test run in Bitbar Cloud

  1. Create Appium iOS Server Side type test run.

    image3

    image4

  2. Upload test zip file and the actual app file (Runner.app ) from build/ios/iphoneos/ -directory converted to an ipa file.

    image5

  3. Select devices

  4. Make sure to set long enough test time-out period

  5. Run tests

Simplified run-tests.sh example for Android integration tests

Example of complete “run-tests.sh” file for Android integration test with minimal content.

unzip tests.zip
yes | flutter doctor --android-licenses
cd flutter_app
flutter clean
rm -r .packages
rm pubspec.lock
flutter drive --target=test_driver/main.dart