Who cares about testing?: Co-creations of Socio-technical Software Testing Experiences

Abstract: Software testing is crucial for ensuring software quality, yet developers' engagement with it varies widely. Identifying the technical, organizational and social factors that lead to differences in engagement is required to remove barriers and utilize enablers for testing. While much research emphasizes the usefulness of testing strategies and technical solutions, less is known about why developers do (not) test. This study investigates the lived experience of software developers to illuminate how their opinions about testing change. Learning about personal evolutions of practice, we explore when and why testing is used. Employing socio-technical grounded theory (STGT), we construct a theory by systematically analyzing data from 19 in-depth, semi-structured interviews with software developers. Allowing interviewees to reflect on how and why they approach software testing, we explore perspectives that are rooted in their contextual experiences. We develop eleven categories of circumstances that act as conditions for the application and adaptation of testing practices and introduce three concepts that we then use to present our theory of emerging testing strategies (ETS) that explains why developers do (not) use testing practices. This study reveals a new perspective on the connection between testing artifacts and collective reflection of practitioners, and it embraces testing as an experience in which human- and social aspects are entangled with organizational and technical circumstances.

• The paper presents a theory of emerging testing strategies influenced by socio-technical factors using a grounded theory approach.
• The study employs semi-structured interviews and iterative analysis to reveal how project complexity, organizational mandates, and cultural beliefs shape testing practices.
• Findings suggest that effective testing requires integrating human experiences with technical practices to foster reflective and adaptive testing cultures.

Co-creations of Socio-technical Software Testing Experiences

Introduction

The study presented in "Who cares about testing?" (2504.07208) explores the varied engagement of software developers with testing practices, emphasizing the socio-technical factors influencing these behaviors. Utilizing Socio-technical Grounded Theory (STGT), the research seeks to unravel the lived experiences behind developers' decision-making processes regarding software testing. By systematically analyzing interview data, the authors construct a theory around emerging testing strategies that encapsulates the complex interplay of social, organizational, and technical conditions.

Socio-technical Grounded Theory Approach

STGT is leveraged as an effective framework for this exploratory study, given its focus on understanding socio-technical phenomena without preconceived notions. The iterative data collection and analysis inherent in STGT allows for the construction of novel concepts and connections between various influences on software testing practices. The work employs semi-structured interviews to provide developers with a stage to reflect on their experiences, thus capturing authentic insights into the socio-technical dimensions of software testing (Figure 1).

Figure 1: Illustration of the first phases of our STGT research design.

Key Influences on Testing Practices

Socio-technical Aspects

The authors identify socio-technical aspects as significant influences on software testing behaviors. These include project complexity, development processes, and notions of safety and responsibility. Complexity, while often perceived as overwhelming, can motivate testing as a means to achieve modular understanding of intricate software systems. Similarly, the structure of the software development process, whether agile or otherwise, determines how developers integrate testing as part of their workflow.

Affordances and Mandates

Affordances refer to the tangible resources and tools available that facilitate testing practices, such as infrastructure and domain-specific approaches. Business demands and application domains shape testing priorities, often dictating where developers allocate resources. Furthermore, organizational mandates can impose testing requirements, although these may be met with resistance if developers perceive them as hindrances rather than enablers.

Dogmatic Perspectives

The research uncovers how ingrained cultures and community perspectives impact testing practices. Developers may hold strong convictions about the utility or burden of testing, informed by the broader community and their previous experiences with testing methodologies. Such cultures within developer communities and projects can significantly shape the testing strategies adopted and how they evolve over time (Figure 2).

Figure 2: Illustration of the last two phases of our research design. We reach the advanced stage when data collection and analysis in the basic stage allows us to establish links between preliminary categories.

Theory of Emerging Testing Strategies

The study proposes a theory that portrays testing strategies as emergent phenomena influenced by a recursive process involving {"testingSignatures"}, {"testingEchoes"}, and {"testingEfficacy"}. TestingSignatures represent the tangible artefacts of testing, whereas TestingEchoes encapsulate the conversations and shared reflections within a team. TestingEfficacy determines the perceived value and efficiency of testing tasks. This cyclical interaction fosters continuous adaptation of testing approaches within the socio-technical environment of a project (Figure 3).

Figure 3: Diagram illustrating the mutual, circular influence of {TestingSignatures} and {TestingEchoes}.

Implications and Future Directions

This research challenges traditional views of software testing as purely technical practice by highlighting the socio-technical dynamics that govern its application. It suggests that effective testing strategies must account for the human and social aspects that interplay with technical considerations. Practically, this points to the need for fostering reflective cultures where developers can negotiate testing practices collaboratively. The findings also prompt further investigation into how testing artefacts and social interactions codify and influence organizational practices.

Conclusion

``Who cares about testing?'' offers profound insights into the socio-technical layers of software testing experience. The theory of Emerging Testing Strategies developed in this study opens up new avenues for researchers to explore the intricate dynamics of software testing practices and provides practitioners with a framework to evaluate and innovate their testing cultures. The research underscores the importance of engaging with developers' lived experiences to better understand and promote effective testing practices that embrace the full spectrum of influences in the software development lifecycle.

By concluding statements in reflective style, the research extends an invitation for continued exploration and refinement of socio-technical grounded theory in the field of software engineering.