Regression test suites are maintained by software system developers to ensure that any new code development does not affect existing functionalities. Such test suites need to be optimal in size so as to balance the requirements of maximum coverage and minimum execution time. For software with multiple input parameters and configurations, Combinatorial Testing (CT) is a method which can be used to generate regression test suites. Regression test selection, augmentation, prioritizing and pruning are the areas in which significant amount of research work has already been done. In spite of this, for most software systems, where multi parameters are involved either in the configuration or in the input parameters of the regression test suite the combinatorial testing is not explored. We explore the combinatorial testing methodology and apply it to regression test suite of case study operating system Contiki in this thesis. Thesis starts with the test suite execution timing analysis. Thesis then proposes an integrated test environment approach which brings about a better integration of different tools which are available. The thesis continues with a demonstration of generating effective test suite for multiparameter software using Advanced Combinatorial Testing for Software (ACTS) Tool and its verification using Code Coverage Tools. The thesis details generation of the regression test suite using a methodology called as CTS based Regression Test Suite (CT-RTS). This is applied to a free software called College Time Table (CTT). To demonstrate the advantages of use of CT-RTS the thesis then does a detailed study of Contiki, an IoT Operating System. The Internet of Things technology deals with connecting devices, called as things, to Internet via standardized networking protocols. The networking protocol is part of the Operating System (OS) deployed on the nodes or motes of Internet of Things. In addition to networking capabilities, the OS needs to meet the requirements of extended battery life, memory constraints etc.. Since it is an evolving piece of software, the testing of the OS has to be thorough and streamlined. Details of the existing regression test suite of Contiki is presented. The thesis lists some of the limitations of the existing regression test suite by studying the code coverage. To overcome these limitations, the thesis proposes two methods for generation of test suite using CT-RTS: One is to augment the existing test suite. Second mechanism is to create a new test suite. The CT-RTS can be used for customizing the number of tests incorporated in the test suite based on the coverage required. The thesis shows the effectiveness of the CT-RTS by applying it to Contiki Operating System and its Cooja simulator. The utility of the CT-RTS has been demonstrated by solving a research problem in the area of Regression test suite creation. The CT-RTS used to create a functional Regression test suite creation mechanism which can be used for large multiparameter software. The thesis shows that it is possible to take a more rigorous approach to the problem of Regression test suite creation using the CT-RTS approach. The thesis makes the following research contributions. 1. Study of generic regression test suite is done in this phase. Regression test suite execution timing analysis methodologies are studied using the statistical approach. 2. An integrated test environment approach for combinatorial testing is proposed. It is a centralized approach which reduces the number of tools and duplicated functionality leading to better integration of different tools making maintenance of the test setup simple. 3. Thesis demonstrates generation of a regression test suite for a multi parameter software using a proposed methodology called CT-RTS. The effectiveness of the test suite is verified using the traditional code coverage metrics. 4. Thesis proposes the test design methodology for Internet of Things operating systems. Extraction of parameters by studying the existing regression test suite, the execution and use of the software is demonstrated. 5. CT-RTS approach is applied to generate regression test suites for the Contiki OS and Cooja simulator. Comparison of the effectiveness of the existing test suites, re-architectured test suites and new test suite designed using CT-RTS is done. 6. The thesis proposes a mechanism for automation of test scripts generation. This has been successfully used to generate functional test suites for Contiki OS and Cooja simulator. 7. Residual test coverage algorithm is enhanced for prioritization of the regression test suite using the code coverage and execution time as the input parameter. Further, black box approach to test suite prioritization using statistical techniques is proposed.

TITLE - An Attempt to Design and Implement Combinatorial Testing based Test suites for Operating Systems used for Internet of Things
AUTHOR - Abhinandan H. Patil
••••••IJSER Edition - February 2019

UNIVERSITY - BITS Pilani, Goa
GUIDE NAME -

Krishnan Rangarajan