Table of Links
Abstract and 1 Introduction
2.1 Software Testing
2.2 Gamification of Software Testing
3 Gamifying Continuous Integration and 3.1 Challenges in Teaching Software Testing
3.2 Gamification Elements of Gamekins
3.3 Gamified Elements and the Testing Curriculum
4 Experiment Setup and 4.1 Software Testing Course
4.2 Integration of Gamekins and 4.3 Participants
4.4 Data Analysis
4.5 Threats to Validity
5.1 RQ1: How did the students use Gamekins during the course?
5.2 RQ2: What testing behavior did the students exhibit?
5.3 RQ3: How did the students perceive the integration of Gamekins into their projects?
6 Related Work
7 Conclusions, Acknowledgments, and References
2.1 Software Testing
Software developers have access to various tools and metrics that help them identify how to improve their tests, such as code coverage, which identifies statements, branches, or execution paths that have not been covered by tests. This measurement is achieved by recording the parts of the application that are reached during test execution, resulting in lists of uncovered lines of code, percentages of covered code, and visualizations of the coverage [53]. Mutation analysis [28] assesses the effectiveness of test suites by inserting artificial faults (mutants) into a program and checking whether the tests can detect them and provides guidance on additional tests if not. Static code analysis is used to identify common patterns in the code that may lead to potential bugs. For example, patterns such as code [17] and test [48] smells suggest where code should be refactored or removed by developers to improve code quality.
Although there are various metrics and tools for software testing, they are often not widely used in an automated manner [6]. Continuous integration (CI) is a common solution to this problem: CI allows developers to automatically execute their tests and calculate metrics such as code coverage after each push to a remote repository [8]. This enables developers to focus on writing code and tests in their IDE, while the CI system performs analyses and provides a summary of findings and uncovered code [44]. The use of CI platforms has been shown to improve productivity and enable faster software releases with higher quality [43, 52].
2.2 Gamification of Software Testing
The lack of motivation to write tests and to use test automation can at least partially be attributed to the perception of testing as being tedious, stressful, uncreative [30, 49, 50], or not appreciated [13, 14]. Motivation, which is crucial for developer productivity [18–20], can be intrinsic, describing the inner willingness to participate in an activity, or it can refer to an external signal related to a task. Gamification is a way to provide external motivation, and it does so using game design elements within non-game contexts [15]. Gamification has been shown to increase engagement and improve results compared to non-gamified development environments [11, 45]. The most common gamification elements include points, badges, and leaderboards, but other elements such as challenges and achievements are also often applicable [39].
In the context of software testing, gamification has been applied in both professional (e.g., [37, 40]) and educational (e.g., [5, 21, 22, 51]) settings. Previous attempts to gamify software testing education have focused on specific aspects such as mutation testing [21] or security testing [22]. Our aim is orthogonal: Rather than teaching specific test techniques, we aim to get students to integrate testing into their regular software development routine.
Authors:
(1) Philipp Straubinger, University of Passau, Germany;
(2) Gordon Fraser, University of Passau, Germany.
