In their recently published work [1], HASEL members James Tizard, Peter Devine, Hechen Wang, and Kelly Blincoe, studied how user engagement with online support forums is used by development teams to improve their software.
User opinions in online feedback are a vital resource in modern software development, helping to give insights into user needs, and usability issues. These user generated insights can be documented as requirements for the software (in an issue tracker), which can then be used to guide product improvement. Keeping users happy by continuously addressing their feedback is critical for a modern product’s success, as for many applications the competitor is only one click away.
In support forums, users generally describe issues they’re having with software, and are seeking help to resolve. Their forum posts have previously been found to contain information that can be leveraged for software improvements, such as bug reports, unintuitive features, or missing features. However, it was previously uninvestigated if this forum feedback was, in fact, being used in software development.
In their new study, the HASEL group performed an empirical analysis of the forums of two popular open-source products, the VLC media player, and the Firefox web browser. In both products, the researchers found that project contributors do, in fact, manually evaluate, and act on, the issues described in forum posts. They observed when a forum post describes a new, previously unknown issue, project contributors will manually transfer it to an issue tracker, for development attention. Additionally, they saw when the posted issue is already known, project contributors will support the user by directing them to documentation, containing a detailed solution, or workaround for that issue.
Manual forum management by project contributors, as described above, undoubtedly requires considerable time and effort. Therefore, next the HASEL group proposed state-of-the-art deep learning (DL) techniques that could help automate the observed behaviour. Applying the DL model USE, they found new forum posts can quickly, and with promising accuracy, be compared to all existing issue tracker entries, to assess if a post represents a new software requirement. Applying this technique could remove the need for project contributors to manually evaluate the novelty of each new forum post. Additionally, when the user issue is known, applying USE again, the researchers found forum posts can be quickly matched, with promising accuracy, to the most relevant solution, or workaround, for that issue. In this way forum users could quickly be supported, without the need for human intervention.
For complete details of this work, please check out the full paper, here! [1] There is also a full replication package of the proposed DL analysis techniques, available at https://figshare.com/s/6b583d1882bb3f1baa1b.
[1] Tizard J, Devine P, Wang H, Blincoe K. A Software Requirements Ecosystem: Linking Forum, Issue Tracker, and FAQs for Requirements Management. IEEE Transactions on Software Engineering. 2022 Nov 7.
Recent Comments