How should self-deprecation comments be classified? A toxicity analysis study on Zephyr

Abstract

Background: Toxic language in Open Source Software (OSS) code reviews can harm contributor well-being and retention. Detecting toxicity is especially challenging for context-dependent language, such as self-deprecation, expressions that may include strong terms (e.g., "damn, I need to look more carefully at these"), but are self-directed and non-abusive against other contributors. Aims: We investigate how to distinguish self-deprecation from interpersonal toxicity in OSS code reviews. Method: We present a study on the Zephyr Real-Time Operating System (RTOS) project, examining how self-deprecation appears in code review comments and how software engineering specific toxicity models handle it. Using a toxicity detection tool across pull request review comments, we retrieved the messages classified as toxic. We then combined a Large Language Model-based, context-aware judgment with targeted human validation to identify and categorize self-deprecating comments and qualitative analysis to find different types of self-deprecating comments. Results: Our analysis of over 272k comments revealed 2,097 toxic comments, and from those, we classified the self-deprecating toxicity comments into apologies, self blame, self derogation and self expletives. Conclusions: We identify self-deprecation as a distinct, common pattern that merits differentiated treatment from interpersonal toxicity. The proposed four-type taxonomy can guide configurable moderation settings and more nuanced classifiers that reflect project norms.