Building a Smartphone Test Platform by Using RTK and Open Source Software
Building a site for mobile testing platforms with Jenkins, Ruby, and RTK
Hiroki Maruchi and Shoichi Iwai, GBA Co., Ltd.
Table of Contents
- Testing an RTK Rental Device with a Jenkins Job
- Checking the Screen on the Build Result
- Further Application: Improving Test Development "Continuously"
In one of the previous articles, "Using Jenkins for the First Time," we installed Jenkins, and ran a job.
In the previous article, "Programming a Smartphone Test in Ruby," we used RTK Thrift API and from the test script created by the user, we rented a smartphone on RTK and ran an app on the device. We then captured the screen of the run result.
This article sums up that procedure. The system configured by the laptop PC in the below image uses several smartphones in RTK, and runs the app build and the test.
2. Testing an RTK Rental Device with a Jenkins Job
In the section "Creating a Job: Build an Android Studio Project" in one of the previous articles titled "Using Jenkins for the First Time," we built an Android project and installed it on an emulator.
In the previous article, "Programming a Smartphone Test in Ruby," we used the RTK Thrift API feature and accessed RTK via the Internet with the Ruby test script. We also rented a smartphone in the RTK cloud and controlled it to install an app as well as capture screens of the Web and the app.
Here, we combine those two procedures and create a Jenkins job to run a test on a smartphone that uses RTK. The following is an example of the job setting values. Basically, we are only adding the content of the test script for the app build to the content we introduced in the previous article.
Creating a job
Create a new job.
Input a job name and select Freestyle project.
Here, we named the project "test_calculator_004".
Specifying the source code location
Advanced project option
> Use a custom workspace… Check
> Directory field: /Users/hiroki/AndroidStudioProjects/calculator_002
> Run the shell
> Command field: /gradlew build # Run the build
> Run the shell
> Command field: ruby test_004.rb # Run the RTK test
> Save the result
> Field for the file to be saved: **/*.apk
Once setup is complete, click the Save button to save the job.
Running a job
Click Build Now. The job runs.
While the job is running, you can display content on the console. Finally, Finished: SUCCESS is displayed and the job completes running successfully.
Displaying the result
In the same manner as previously, after the job completes, the result of whether the job ran successfully or not is displayed using green and red traffic lights.
The image of the captured result is saved in the project directory "app/build/reports/rtk".
Thus, you can confirm whether the app ran successfully on the test smartphone without the intervention of a person. However, the screen capture of the result is only placed on the local directory of the development PC. So that all members can check from Jenkins, we would like some more features.
3. Checking the Screen on the Build Result
Job Modification: Checking the Screen on the Jenkins Web Site
For the Jenkins job in the previous section, we ran an app on the smartphone we rented on RTK and locally downloaded the run result as a screen file. However, the screen file that is the test result resides on the local PC.
This time, let's check the result screen file on Jenkins.
To do this, all we have to do is save the image to Save the result in the settings.
> Save the result
> Field for the file to be saved: **/*.apk, **/capture_*.jpg … Also make sure to save the images.
In the same manner as previously, let's run the job.
The capture of the result will also be displayed as an item in Build Artifacts.
You can display the image of the result in a browser by clicking the link.
Simply put, previously, you could only view the result on the local PC, but now you can share the result with other Jenkins members as soon as the job completes.
4. Further Application: Improving Test Development "Continuously"
In the section above, we showed that we can perform smartphone app development in any environment by using only a small number of the RTK Thrift API features. Depending on the type of development and the team, there are almost limitless demands. It goes without saying that all the features cannot be introduced here. We give you some tips as follows.
Using the RTK Thrift API features
The following basic features are available in RTK Thrift API:
- Device rental/automatic extension feature
- Web page capture
- Virtual adb, Wifi, GPS settings
- Screen capture, device rotation
- Video recording
Only a small number of features were used in this example. The full operation can be recorded as a video as well to meet the purpose the user is developing the app. Furthermore, RTK Thrift API is currently being actively developed. In the near future, features will be added and it will be improved. Please check the RTK Web site for added features and specifications.
Using Jenkins' features
Builds and tests are run every day and git commit can be run as an event. This can be set comparatively easily in Build Triggers and Source Code Management in the Jenkins job settings.
In this article, the developer clicks a button to start builds and tests. By converting to the procedure stated above, the developer does not need to be involved manually and specialization can be avoided. Additionally, when making a commit or a change, you can always check whether it passed or failed, and you can introduce quality engineering-type methods, such as regularly performing multiple tests, gathering statistics, and so forth.
Using the user test script feature
If you want to add a feature that is not in RTK Thrift API or Jenkins, there are times when you can use the user test script to add that feature.
For example, you can specify the model to rent on RTK, test various models individually, and so on.
In the user test script indicated in this example, there is a filter method to easily select a device to rent (select_device method). We think that this will be helpful for you. Moreover, in order to test the specified number of devices, by including a loop, subsequent tests can also be created relatively easily.
In any case, in order for the entire development team to improve testing, it is important to use a programming language for the user test script that is easily read and maintained by every member including app developers and testers. In this example, Ruby is used. Ruby is used widely in other open source development support software like Chef and ServerSpec as well and we think it is a good option.
In this article, we created a simple system that automates the process up until the testing of the smartphone by using RTK and open source software multiple times.
RTK provides a system in the development process in which manual intervention in test processes involving an actual device are unnecessary. In this article, we were able to relatively easily support actual device testing by just adding test script to the Jenkins scenario. Like this example, we can now easily embed in the automated test process the device testing that we could not perform because it required manual operations in the past. By using RTK, the precision of automated testing increases, and output can also be increased without spending much on setup costs.
In smartphone app development in the past, at the phase testing the final product, a large number of samples were used and a great deal of blood, sweat, and tears were expended in the final check before delivery. This achieves a simple shape like a picture. We believe that this shows that the manufacturing of smartphones can be greatly changed by RTK.