Test Automation and Remote TestKit

1. Introduction

Do you enjoy writing programs?
It is a safe assumption that most people reading this article write programs in an occupational capacity, and engage in software development in some manner on a daily basis. Many programmers who code every day enjoy the thrill of creating and learning new things as they work.
But do you enjoy testing programs?
It would be no surprise if many of you responded in the negative to this question. In fact, common responses would most likely include an enthusiastic "I hate it," or "it's tedious," or "it's painful." Due to the complexity and importance to society of today's software, however, one would be hard-pressed to find an engineer willing to assert that testing is unnecessary. Engineers will often say that they fully understand the importance of testing, and that they do it because programming is not simply something to enjoy, but rather a job with responsibilities.
What is the source of this discrepancy? Also, is there any way to make this tedious testing work any more like the programming work everyone enjoys so much? Furthermore, is there any way to increase the efficiency of testing work, which cannot be said to have a high level of productivity, but rather tends to be more labor-intensive than programming?
This article will provide hints regarding how to test software more efficiently, more proactively, and perhaps even more enjoyably. In addition, this article will show how Remote TestKit consolidates cutting-edge smartphone app development technology into an indispensable tool that enables smooth development at a high level of efficiency.

2. The Software Development Process and Testing

Software development mainly proceeds based on methods such as the following, with testing used to evaluate the quality of the resulting code:

• V-model
• Test-driven development (TDD)

1) V-model
The v-model is a traditional software development technique.
This model splits development processes into design, implementation (coding), and testing phases.

Although this is a famous model with an extensive track record, it separates design and test processes and requires a comparatively large amount of resources, which causes the scale of the project to grow as well.

Since the process that enables developers to grasp bugs and other problems is kept separate from the process that corrects them, it also takes a long time before correction occurs. In other words, the "feedback loop" for correcting problems is long, and the speed of responses is slow.

Fig. (V-Model)

3. Test-Driven Development (TDD)

This technique began to be adopted in 2010 as a means of implementing agile development.
The key point of TDD is to "test first." This means that code for testing is to be created before the production code is implemented (coding). The TDD development technique is based on this "test first" philosophy. The procedure is as follows:

1. Create a to-do list from the functions that must be implemented.
2. Create test code from the to-do list.
3. (Red) First, implement blank code. Of course, testing at this point will result in errors.
4. (Green) Write and implement the minimum code necessary to eliminate the errors that occur in the aforementioned testing.
5. (Refactoring) Review the details of the minimally implemented code and rewrite it in a more suitable manner.

With TDD, rules are first established for suitable behavior of the code that is implemented as objects, and then work proceeds to create and implement code that satisfies these rules. The explicit reference to "objects" here reflects the fact that this technique has a high affinity for object-oriented programming, and is comprised of "small feedback loops" that pair code implementation with testing. This can greatly reduce the response time required to correct any problems in a given area.

On the other hand, with TDD, the programmer must create test code, and this increases the required man-hours at the start. Furthermore, since the object-oriented mode of thinking is extensively employed, many programmers are confused and struggle with the need to adjust to this mode of thinking and culture. Of course, areas that are difficult to deal with through programming will still be difficult to manage even if TDD is used. In general, TDD is adopted for unit (module) test areas in V-model development. "Areas that are difficult to deal with through programming" refers to the range of areas not controlled by the computer, such as physical objects, people, and so on. Specifically, this means cell phones and smartphones that cannot be accessed without the intervention of a person, and work processes that are dependent on individual persons.

Fig. (Steps in Test-Driven Development [TDD])

4. Future Testing and Remote TestKit

This TDD loop configuration may expand in scope to the type of control theory used in the development of robots and other such devices, with the loop going from unit testing to integration testing/functional testing, and then to system testing. This can also be seen as an example of applying the cybernetics (supplemental explanation) advocated by Norbert Wiener to software development.
Recently, continuous integration (CI) tools such as Jenkins (supplemental explanation) have been garnering attention.
This tool can be used as a hub for further automation of compiling or for linking with Selenium (supplemental explanation) or other tools to automate the development of all types of sites and apps.
Remote TestKit, on the other hand, is a service that can be used to access actual smartphones over the network.
This service provides a convenient way to create a test environment that includes many smartphones, even when they are not close at hand. The benefit of this is that developers can test their products on actual smartphones without the costs associated with preparing many devices, and without the need to set up all of the devices while dealing with space limitations. The ability to access the user interfaces of actual smartphones over the network opens up even greater possibilities than this.
In other words, by linking all of the aforementioned items, it is possible to network almost all of the items related to development:

• TDD as a software-based technique
• Various types of CI tools, connected over the network
• Actual smartphones, the user interfaces of which can be controlled over the network

In addition, the scope of test automation can be expanded to include actual devices as well, which would have been difficult to include in the past.

5. Topics Introduced in Future Articles

Subsequent articles will introduce how to use various specific open source CI tools, along with techniques for linking these tools with Remote TestKit in order to construct an automatic testing ecosystem.

Glossary

Cybernetics
An academic discipline advocated by Norbert Wiener (1894?1964) that combines communication engineering and control engineering for uniformly dealing with a wide range of fields including mechanical engineering, systems engineering, physiology and sociology.
Jenkins
An open source continuous integration tool.
CI: Continuous Integration
Continuous execution of creating builds, tests, documentation etc. and improving the quality of various development work and manufacturing.
Selenium
A tool for testing web applications. This tool can remember the browser's operation and execute it.
Nomad
A nomad. A work style of being able to work at various locations outside an organized office by utilizing IT.
Deploy
Setting up developed software in the target environment so it is ready to run. In the broad sense, it is similar to installing software with the added meaning of preparing various settings for the entire target system that includes related modules, configuration files, and access restrictions included.

References

• Junichi Takahashi, "Learning Software Testing from Zero Knowledge," Shoeisha (in Japanese)
• Kazuhiro Ishihara and Masahiro Fuse, "The Easiest Software Testing Book," Gijutsu-Hyohron Co., Ltd. (in Japanese)
• Kazuhiro Ishihara and Hidekazu Tanaka (authors), Shinji Tanaka, "Software Testing Textbook," SB Creative (in Japanese)
• Kenji Onishi, "Practical Guide to Software Testing," Nikkei Business Publications (in Japanese)
• Mark Fewster, Dorothy Graham et al., "Software Test Automation," Shoeisha
• Technologic Arts Co., Ltd., Yoshihide Nagase et al., "Test-Driven Development," Tokyo Denki University Press (in Japanese)
• Steve Freeman, Nat Pryce, "Growing Object-Oriented Software, Guided by Tests," Shoeisha