Anaphora Resolution in Software Requirements Engineering: A Comparison of Generative NLP Pipelines and Encoder-Based Models

Abstract

In the field of requirements engineering (RE), anaphoric ambiguity can negatively impact the quality of requirements and could even threaten the success of a project. If different stakeholders like testers or customers interpret software requirements differently, the system might fail to pass the customer validation stage. On the other hand, a robust anaphora resolution model clarifies the writing process of requirements by accurately indicating the pronoun references. In this study, we exploited the power of generative NLP pipelines and compared their performance with the extractive Question Answering (or sequence labeling) technique. We conducted extensive numerical experiments including text-to-text pipelines and compared them with encoder-based models on two public requirements datasets. Our experiments revealed that a sufficiently large T5 model can yield better results than encoder-based models. We've utilized methods such as Lora to effectively address the complexity of training large language models. Our study indicated that the generative approach outperforms classification-based models for anaphora resolution tasks in Software Requirement texts. © 2024 IEEE.