Abstract:
In modern software development, requirements ambiguity is a key factor that leads to project failure, cost overruns, and quality issues. Therefore, the automatic detection of requirement ambiguity has been extensively studied. Although these methods reduce the time cost of manual review, their solutions are incomplete for the types of ambiguity detection, and it is difficult to fully cover the six types of ambiguity corresponding to language ambiguity. The traditional method lacks the ability of in-depth semantic understanding, logical relationship recognition and quotation/referential relationship recognition, which limits its semantic processing ability and cannot detect pragmatic ambiguity and linguistic error ambiguity. To this end, we propose a large language model-based method called LMAdetect for automatic detection of requirements ambiguity. For a given requirement, LMAdetect analyzes the requirement according to the heuristic rules and the large language model, and classifies the requirement detection into the corresponding ambiguity type. However, large language models often classify different types of ambiguity, and for the classification results of large language models, LMAdetect uses a rule-based algorithm to assign different types of ambiguity to different experts for detection. Finally, the detection results are summarized according to the confidence level, and the classification results with high confidence are output. Experiments on
1192 data of 6 ambiguity types show that compared with the traditional rule-based method, the
F1 score of LMAdetect is increased by 34.4%, and compared with the method based on large language model, the
F1 score is increased by 31.6%, and the
F1 score of pragmatic ambiguity and linguistic error ambiguity detection reaches
0.6950 and
0.8889, respectively, demonstrating its advantages in requirement ambiguity detection.