How to Use Remote TestKit | Mobile app testing -Remote TestKit

Remote TestKit

Device Cloud for
Mobile App Testing

Tutorial for Appium Features

Overview

Appium is a test framework to automate testing on a smartphone or tablet.

Previously, automated testing with Remote TestKit was possible using an Appium environment built on a local computer by using the virtual ADB and Xcode connector features. In June 2018 release, Appium test script can now run easily without building an Appium server environment and a link can also be easily made with CI environment.

* Remote TestKit Appium Cloud is available only to customers who have a Flat 3 or higher plan.

Appium Links Supported by Remote TestKit

An Appium Environment that Does Not Use Remote TestKit

Generally, a run environment, such as Node.js or an Appium server, must be built on a local computer that runs Appium, the test script created and then run in order to automate testing with Appium, as shown in the figure below.

Moreover, only a test device connected to a local computer could be used as the device used for the test script and in many cases an Android/iOS simulator had to be used.

An Appium Environment that Uses the Conventional Remote TestKit Features

In the past, it was possible with Remote TestKit to build an Appium environment that used a device on the cloud by using the virtual ADB and Xcode connector features. In this configuration, the link between a cloud hosted device and local environment is established at lower layer, which has the merit of enabling links to test frameworks other than Appium.
Note: Some features are limited in debuggers that use Xcode in iOS devices.

An Appium Environment that Uses New Remote TestKit Features

By using the features in the June 2018 update, automated testing using Appium is possible without building a Node.js, Appium server, or similar run environment. The only thing user needs to prepare is test script. Moreover, by using this feature, links to a CI environment inside or outside of a company can easily be achieved. You can focus on the task of creating test script. An Appium environment that uses the new features is shown below.
Note: At the time of this release, only devices running Android 4.4 or later and iOS 10.3 or later are supported.

Comparison of Appium Environments

The following chart compares the three environments listed above.

Environment that uses new RTK features Environment that uses conventional RTK features Environment that does not use RTK
Building of a server environment Not required Required Required
Client library Required Required Required
Physical Test smartphoneDevices Not required Not required Required
Test script run location Local computer Local computer Local computer
Linksing to toolss other than Appium Not supported Supported Supported
Difficulty of Integration with CI linking Easy Linking using the Devices need to be used through Thrift API is required Difficult

Resolving Issues

One of the challenges implementing automated testing within Continuous Integration (CI) tool is testing on physical devices. By using the new Appium features, the test script can now run on physical devices from CI services comparatively more easily than in the past.

Appium test script can run on devices within Remote TestKit from Jenkins or other CI environments built in-house.

How to Connect Appium Using the New Features

By just changing the Capabilities settings for the test script running in the Appium environment on the local computer, the server-side Appium features can be used.
The following is an explanation using an example test script written in Ruby. First of all, prepare a PC that Ruby can works.

Installing Appium Client Library

Follow the procedure below to install the Appium client library.

gem install selenium-webdriver
gem install appium_lib

Explanation of the Destination

When running a test script on a local computer, specify "https://gwjp.appkitbox.com/wd/hub" for "http://localhost:4723/wd/hub" as the destination for the Appium client (test script).

・Example Ruby code

    @driver = Appium::Driver.new(
    caps: {
        ...
    },
    appium_lib: {
        #server_url: 'http://localhost:4723/wd/hub',
        server_url: 'https://gwjp.appkitbox.com/wd/hub',
        wait: 60
    }
).start_driver

User Authentication

Specify Remote TestKit user name and password or access Token in Capabilities to connect to the Appium client. Also, the access token can be generated from Remote TestKit Software.

Capabilities name Value
userName Remote TestKit user name
password Remote TestKit password
access Token Remote TestKit access Token

・Example Ruby code (authenticate with username and password)

@driver = Appium::Driver.new(
    caps: {
        userName: "Remote TestKit user name",
        password: "Remote TestKit password",
        ...
    },
    appium_lib: {
        ...
    }
).start_driver

・Example Ruby code (Authenticate with accessToken)

@driver = Appium::Driver.new(
    caps: {
        accessToken: "Remote TestKit access Token",
        ...
    },
    appium_lib: {
        ...
    }
).start_driver

・How to get the access tokens

The access tokens can be generated from Remote TestKit software. Click the "Manage access token" right side menu. To regenerate the access token, click the regenerate button.

Selecting a Appium version

For Android devices, Appium 1.7.2,1.8.0,1.8.1,1.9.0,1.9.1,1.10.0,1.10.1,1.11.0,1.11.1,1.12.0,1.12.1,1.13.0,1.14.0,1.14.2,1.15.1,1.16.0,1.17.0,1.17.1,1.18.0,1.18.1,1.18.2,1.18.3,1.19.0,1.19.1,1.20.0,1.20.1,1.20.2,1.21.0 runs on Remote Testkit, Appium version specification is possible with capability, default is 1.8.0.
For iOS devices, Appium versions are determined automatically, so capability can not be specified.Please make sure the test script written on your side is compatible with supported versions on Remote TestKIt.

Capabilities name Value
appiumVersion Specify from "1.7.2,1.8.0,1.8.1,1.9.0,1.9.1,1.10.0,1.10.1,1.11.0,1.11.1,1.12.0,1.12.1,1.13.0,1.14.0,1.14.2,1.15.1,1.16.0,1.17.0,1.17.1,1.18.0,1.18.1,1.18.2,1.18.3,1.19.0,1.19.1,1.20.0,1.20.1,1.20.2,1.21.0"

Selecting a Device

The device to run Appium can be selected by the device name only or the device name and version. Regular expression can also be used to specify the device name.

Capabilities name Value
platformName Specify "Android" or "iOS"
deviceName Specify the device name
Determined in the following order: "Match all" -> "Match front with space delimiter" -> "Match front regular expression" (not case sensitive)

・Example Ruby code (with Nexus 5 specified)

@driver = Appium::Driver.new(
    caps: {
        ...
        platformName: 'Android',
        deviceName: 'Nexus 5X',
    },
    appium_lib: {
        ...
    }
).start_driver
Capabilities name Value
platformName Specify "Android" or "iOS"
deviceName Specify the device name
Determined in the following order: "Match all" → "Match front with space delimiter" → "Match front regular expression" (not case sensitive)
platformVersion Specify the version
Specify the part excluding "Android" "iOS" in the OS column of the computer client
Determined in the following order: "Match all" → "Match front regular expression"

・Example Ruby code (with Nexus 5 version 8 specified)

@driver = Appium::Driver.new(
    caps: {
        ...
        platformName: 'Android',
        deviceName: 'Nexus 5X',
        platformVersion: '8',
    },
    appium_lib: {
        ...
    }
).start_driver

・Example Ruby code (with iPhone 7 version 11.1 specified)

@driver = Appium::Driver.new(
    caps: {
        ...
        platformName: 'iOS',
        deviceName: 'iPhone 7',
        platformVersion: '11.1',
    },
    appium_lib: {
        ...
    }
).start_driver

Installing the App

Method 1.

Upload apk or ipa file to a public server which can be accessed from Remote TestKit and specify the Capabilities below.
Note: If necessary, add settings to permit access from Remote TestKit to the internal firewall. The access source IP address is listed in the FAQ.

Capabilities name Value
app Specify the URL of the uploaded apk or ipa
bundleId Specify the bundleId of ipa. Required parameter for ipa.

・Example Ruby code

@driver = Appium::Driver.new(
    caps: {
        ...
        app: "https://appkitbox.com/test/sample.apk",
    },
    appium_lib: {
        ...
    }
).start_driver

Method 2.

Log in to the Software version of Remote TestKit and add the apk or ipa at "Manage application storage" from right side menu. Next, specify the following Capabilities.

Capabilities name Value
app Specify the file name of the uploaded apk or ipa
If multiple files with the same name are uploaded, the latest uploaded file will be used.

・Example Ruby code

@driver = Appium::Driver.new(
    caps: {
        ...
        app: "sample.apk",
    },
    appium_lib: {
        ...
    }
).start_driver

Checking the Test Screen in Progress

After connecting to the Appium server, a snapshotUrl is issued in order to obtain screens. Use Capability to obtain the snapshotUrl. The currently running screen can be viewed by opening the snapshotUrl in a browser.

・Example Ruby code

snapshotUrl = @driver.capabilities['snapshotUrl']

Obtaining the Appium server log

After connecting to the Appium server, a serverLogUrl is issued in order to obtain appium server log. Use Capability to obtain the serverLogUrl. The currently running appium server log can be viewed by opening the serverLogUrl in a browser. The appium log is the latest 10-minutes log. At this time, logs can be obtained only for Android.

Capabilities name Value
logLevel Specify "info", "warn", "error", "debug". The default value is "info"

・Example Ruby code

serverLogUrl = @driver.capabilities['serverLogUrl']

Automatic Return of a Rented Device

If the same device is not accessed for ten minutes via the Appium server, it is automatically returned.

Automatic Extension of a Rented Device

If the same device is accessed via the Appium server with less than ten minutes of rental time left, rental time is automatically extended by 30 minutes.

A Ruby Test Script that Performs a Google Search

Save the following test script with the name of test.rb or the like to the local computer.

require 'rubygems'
require 'test/unit'
require 'selenium-webdriver'
require 'appium_lib'
# get userName, password from Environment variable
RTK_USERNAME = ENV['RTK_USERNAME']
RTK_PASSWORD = ENV['RTK_PASSWORD']
unless RTK_USERNAME && RTK_PASSWORD then
    puts "Environment variable error"
    exit(0)
end
class OpenUrlTest < Test::Unit::TestCase
    def setup
        opts = {
            caps: {
                userName: RTK_USERNAME,
                password: RTK_PASSWORD,
                platform: 'ANDROID',
                deviceName: 'Nexus 5',
                platformName: 'Android',
                browserName: 'Chrome'
            },
            appium_lib: {
                server_url: 'https://gwjp.appkitbox.com/wd/hub',
                wait: 60
            }
        }
        @driver = Appium::Driver.new(opts).start_driver
    end
    def teardown
        @driver.quit()
    end
    def test_google_search
        puts @driver.capabilities['snapshotUrl']
        # Open URL
        url = "https://www.google.com/"
        puts "Open URL: " + url
        @driver.get(url)
        element = @driver.find_element(:name, 'q')
        sleep(5)
        @driver.save_screenshot('capture_01.png')
        # Input keys
        word = "Remote testKit"
        puts "Input Keys: " + word
        element.send_keys(word)
        element.submit
        sleep(5)
        @driver.save_screenshot('capture_02.png')
        # Get value
        value = @driver.find_element(:name, 'q').value
        puts "Text field value=" + value
        assert_equal true, value == "Remote testKit"
    end
end

A Ruby Test Script that Performs a Calculation on iPhone device

Save the following test script with the name of test.rb or the like to the local computer.


# gem install selenium-webdriver
# gem install appium_lib
# export RTK_USERNAME=xxxx
# export RTK_PASSWORD=xxxx

require 'rubygems'
require 'test/unit'
require 'selenium-webdriver'
require 'appium_lib'

# get userName, password from Environment variable
RTK_USERNAME = ENV['RTK_USERNAME']
RTK_PASSWORD = ENV['RTK_PASSWORD']
unless RTK_USERNAME && RTK_PASSWORD
  puts 'Environment variable error'
  exit(0)
end

class ContactsAndroidTests < Test::Unit::TestCase
  def setup
    opts = {
      caps: {
        # get userName, password from Environment variable
        userName: RTK_USERNAME,
        password: RTK_PASSWORD,
        deviceName: 'iPhone 7.*',
        platformName: 'iOS',
        platformVersion: '11',
        bundleId: 'com.apple.calculator'
      },
      appium_lib: {
        server_url: 'https://gwjp.appkitbox.com/wd/hub',
        wait: 60
      }
    }
    @driver = Appium::Driver.new(opts).start_driver
  end

  def teardown
    @driver.quit
  end

  def test_google_search
    puts @driver.capabilities['snapshotUrl']
    @driver.save_screenshot('capture_01.png')

    el1 = @driver.find_element(:accessibility_id, '1')
    puts el1
    el1.click
    @driver.save_screenshot('capture_02.png')

    is_lang_english = true
    begin
      el2 = @driver.find_element(:accessibility_id, 'multiply')
      puts el2
      el2.click
    rescue StandardError
      is_lang_english = false
      el2 = @driver.find_element(:accessibility_id, '乗算')
      puts el2
      el2.click
    end
    @driver.save_screenshot('capture_03.png')

    el3 = @driver.find_element(:accessibility_id, '3')
    puts el3
    el3.click
    @driver.save_screenshot('capture_04.png')

    el2.click
    @driver.save_screenshot('capture_05.png')

    el3.click
    @driver.save_screenshot('capture_06.png')

    if is_lang_english
      el4 = @driver.find_element(:accessibility_id, 'equals')
      puts el4
      el4.click
      @driver.save_screenshot('capture_07.png')

      el5 = @driver.find_element(:accessibility_id, 'Result')
      puts el5
      value = el5.attribute('value')

      puts 'Text field value=' + value
      assert_equal(value, '9')

      el6 = @driver.find_element(:accessibility_id, 'clear')
      puts el6
      el6.click
      @driver.save_screenshot('capture_08.png')

    else
      el4 = @driver.find_element(:accessibility_id, '計算実行')
      puts el4
      el4.click

      el5 = @driver.find_element(:accessibility_id, '結果')
      puts el5
      value = el5.attribute('value')

      puts 'Text field value=' + value
      assert_equal(value, '9')

      el6 = @driver.find_element(:accessibility_id, '消去')
      puts el6
      el6.click
      @driver.save_screenshot('capture_08.png')
    end
  end
end

When running this test script, set the Remote TestKit user name and password in an environmental variable. Use a terminal to run the following command.

export RTK_USERNAME=xxxx
export RTK_PASSWORD=xxxx

Other Sample Codes

For other languages and test script sample codes, see the following repository.

https://github.com/remotetestkit/sample-code

Technical Specifications and Limitations

As of Sep 2018, features are provided as the beta version. At the time of this release, the following are known limitations: