HASEL members, Sanuri Dananja Gunawardena, Peter Devine, Isabelle Beaumont, Lola Garden, and Dr Kelly Blincoe, together with Dr Emerson Murphy-Hill at Google, recently studied software developers’ perception of destructive feedback in software code review.
Software developers have said that code review can create toxic, unsupportive environments and that code review comments can contain judgmental questions, sarcasm, or only negative emojis. We believe that this can reduce inclusion in the software industry. Thus, our aim was to study the impact of such negative feedback and particularly look at differences across genders.
Using an anonymous online questionnaire, we asked participants to consider how they would react to a set of code review comments. Each participant received one constructive code review comment and one destructive code review comment (a destructive comment is both nonspecific and inconsiderate). We also asked participants to answer questions related to the frequency they receive and give destructive criticism and on their opinions of destructive criticism in code review in general. We received 93 complete responses to this questionnaire (43 women, 43 men, 3 non-binary, 4 did not disclose gender).
Overall, destructive feedback was perceived to be less appropriate, less valid, and less likely to help improve the code quality compared to constructive criticism. Compared to men, women and non-binary participants reported to perceive destructive criticism as less appropriate. Open-ended comments from participants demonstrate that there is disagreement between software developers on how code review comments should be written, with some advocating for polite comments while others saying that direct feedback that may appear harsh or inconsiderate is easier to parse and act on. Some participants described the negative impacts they have faced due to receiving destructive criticism. The negative impacts included impacts on team dynamics, individual motivation and confidence, and diversity and inclusion.
Respondents report receiving more destructive criticism than giving such criticism. Of respondents who receive code review comments, 22% of respondents report receiving inconsiderate negative feedback at least once a year and 55% report receiving nonspecific negative feedback at least once a year. There were no differences in likelihood of receiving destructive feedback across genders.
Compared to constructive criticism, destructive criticism negatively impacts motivation to continue working and mood of the participants. Compared to men, women and non-binary participants reported to be less motivated to continue working after receiving destructive criticism.
The findings show that destructive criticism has negative impacts, but that there are conflicting opinions on destructive criticism. Based on our analysis and participant responses, we made recommendations to improve software code review including the creation of mechanisms in code review tools to allow code reviewers to become aware when they provide destructive criticism and better guidelines on when code review feedback on public channels is inappropriate.
This work was supported by a Google Faculty Research Award. The paper reporting the results from this study will be presented at the ACM Conference On Computer-Supported Cooperative Work And Social Computing (CSCW) in November 2022 and will be published in the Proceedings of the ACM on Human Computer Interaction. You can read the full paper here.