Remote Pair Programming’s Hidden Strengths in Advanced CS | HackerNoon

Table of Links

Abstract and 1 Introduction

2 Previous Research

3 Course Description

4 Methods

5 Results

6 Discussion

7 Conclusion and Future Work, Acknowledgments, and References

4 METHODS

4.1 Research Design and Participants

Participants included all Data Structures students. Data was collected from student records, including student examination scores and assignment completion. In addition, we conducted a pre- and post-course survey asking students to self-evaluate the effectiveness of active learning practices (pair programming in detail) and to report on confidence, persistence in the major, and preference. The 4-minute surveys were voluntary and were sent by another instructor to mitigate perceived coercion. The survey instruments were piloted before the semester. Each survey had an 8-day window for completion, with students receiving extra credit on the final exam for successful completion of both surveys. Alternative means of obtaining equivalent extra credit was provided. The pre-course survey had a completion rate of 87% and the post-course survey had a completion rate of 92%.

4.2 Measures

Students completed the first survey (described above) at the beginning of the semester to gather demographic information and capture attitudes toward computer science. At the end of the semester students completed the second survey which included additional questions related to their experience in the course. The pair programming section was also asked questions related to their experiences using pair programming.

Students were given 4 programming assignments (weighted 40%) and 3 exams (weighted 60%) throughout the semester, with other practice assignments not counting toward final course grade. All exams were completed individually. A final score for the class was given based on a composite score of all course work and exams.

4.3 Data Analysis

The primary outcome explored was students’ final score in the class. This score was compared between the pair programming section and the section working individually using an independent samples t-test. Separate repeated measures ANOVA models were estimated to explore exam scores over time, and scores on assignments over time across the two groups. Differences found were further explored using post hoc mean comparisons. The type I error rate in post hoc analyses were adjusted using Bonferroni’s adjustment. Additionally, gender differences and differences between pairs working remotely versus in person were explored.

5 RESULTS

A total of 88 students completed both surveys: 64 male students and 24 female students. Pair programming (n=56) and individual sections (n=32) were similar at both at baseline and end of the term with respect to responses given on the survey with the exception of gender. There were more women in the pair programming section than in the individual programming section (20 versus 4).

5.1 Individual vs. Pair Programming

In Table 1, repeated measures ANOVA on programming assignments showed both a within-subject (p = .003) and a between subject effect (p = .001). The pair programming group consistently scored higher than the individuals over time, with the difference ranging from 8–14% (Figure 1). Additionally, the pair programming group scored nearly perfectly on assignments 3 and 4.

Repeated measures ANOVA on exam score showed a withinsubject effect (p < .001) but no between subject effect. The pair programming group scored higher than the individual programming group as the semester progressed (Figure 2). Final scores revealed a 7% higher score among students in the pair programming group (87.4 vs. 80.4; p = .013) compared to students in the individual programming group (Table 1).

From the survey, for both paired and individual students, there was a slight decrease in the number of students planning “to continue to pursue a career in computing/programming” (Figure 3). This effect was mitigated in the pair programming group, which saw a total increase of 1.8% in those selecting “no” or “unsure”, as opposed to an increase of 12.3% in the individual group.

Finally, individual students were initially more comfortable asking questions in class than pair programming students (Figure 4). However, by the end of the semester, pair programming students had slightly surpassed the individuals in number of students who “strongly agreed” that “I am comfortable asking questions in class” (44.6% vs. 43.8%). This suggests that pair programming may positively affect student confidence related to speaking up in class.

5.2 In Person vs. Remote

Students in the pair programming group were asked to respond to the statement “my pair programming partner and I always worked remotely” on the end of semester survey. Overall, 34 (59%) strongly agreed, 21 (36%) somewhat agreed, 2 (3%) somewhat disagreed, and 1 (2%) strongly disagreed. It appears one pair had a student who

somewhat agreed with the statement, while the other somewhat disagreed. Ultimately, while the course itself was offered in person, most students chose to pair program remotely. While we did not ask the students why they chose to do so, we suspect it was due to convenience (several students mentioned something similar to struggling with finding a common time to meet) or safety concerns (e.g., to protect against the transmission of Covid-19).

Programming assignments and final scores were compared between those who agreed or strongly agreed to working remotely and those who disagreed or strongly disagreed (i.e., worked in person). An interesting trend emerged (Table 2). The T-tests showed no statistically significant difference between both groups on any assessment, except for assignment 4, in which the group that worked remotely did better (by 2.9 points) than those that worked in person (p = .012). In addition, the actual difference between remote and in person groups narrowed over time, suggesting that those who pair programmed remotely found a routine that worked as the semester progressed. This means that no adverse effects of remote pair programming on student outcomes were observed in this study.

5.3 Gender Differences

Women in the pair programming group scored almost 12% higher than women in the individual programming group on their final

score in the course. While this difference did not achieve statistical significance (p = .091) due to the low sample size of women in the individual (non-pair) programming group, a 12% difference is practically significant in that it could result in a difference of 2 letter grades (e.g., from a 79 to a 91).

There were also several gender differences in responses on the end of semester survey from students in the pair programming section. First, women reported an increase in comfort asking questions in class, when compared to men (Figure 5). At the beginning of the semester, only 15% strongly agreed with the statement, “I am comfortable asking questions in class”. At the end of the semester, 40% of women students agreed with the statement, and fewer women than men disagreed with the statement. This is important, because it is an indicator of confidence and implies the ability to handle graceful failure in front of peers. By participating in classroom discussion, there is greater opportunity for learning.

Both men and women saw a similar decrease in confidence in their programming abilities by the end of the semester (a reduction of 10% in “strongly agree”, Figure 6). In contrast to a first year course, where many students are programming for the first time and seeing marked strides in learning a new language, students in a CS-3 course like Data Structures have prior experience, but are then exposed to significantly difficult assignments. The learning curve is much higher. Thus, it is not surprising that some students adjusted their confidence after experiencing the rigor of solving large, real-world problems.

No male student indicated that he had a definitive plan to pursue a career outside of computer science (Figure 7). Some women, however, noted definitively that they did not intend to continue work in computer science. A representative comment that explains the sentiment behind taking Data Structures, but not desiring to continue in the field is the following from a woman in the first section: “I feel as though I am a people person and I would not enjoy sitting behind a computer computing/programming all day. I’d rather sell the product [of] the computation to people, but have the understanding of what it is doing.”

Finally, when asked to respond to the statement “I prefer pair programming to working individually” women had more extreme responses than men. Almost one-quarter (23.8%) of female students strongly agreed compared to 8.1% of male students. Similarly, onethird of women (33.3%) strongly disagreed compared to 18.9% of men. The differences in responses to this question were marginally significant with p = .052.