Extraction of app problems and its corresponding user action from user review of apps using few-shot learning

Abstract

User reviews provide the developers with very important feedback on how to fix the problems, improve application performance, and enhance the user experience. However, these are very unstructured, containing typographical errors, grammat ical mistakes, and informal language in nature, hence making extracting action able insights from such huge reviews a big challenge for developers. The current research focuses on the extraction and synthesis of user behaviors and related ap plication problems extracted from app reviews in the social media industry using large language model capabilities such as those offered by GPT-3.5 Turbo. Our ap proach extends previous ones, like Caspar, by overcoming their major limitations through few-shot learning and advanced prompt engineering techniques. This study shows the need for an effective mechanism to identify and synthesize action-problem pairs from user reviews. We used a dataset of 330 reviews from social media applications to train a fine-tuned model that could handle diverse scenarios, even including reviews without explicit key phrases. Unlike previous attempts, our system effectively captures complex interactions, such as many user activities resulting in a single app fault or one action generating multiple issues. The methodology included substantial data preparation, advanced rapid engineer ing, and performance testing against known baselines. Metrics like recall, and accuracy demonstrate notable improvements over traditional models like Caspar. These results from the current research prove that the proposed model outperforms the state-of-the-art methodologies in the proper classification of action-problem pairs and dealing with noisy, informal data, which is often a very neglected chal lenge. This work shows the potential of deep natural language processing tech niques in software development and maintenance, emphasizing large datasets and automated frameworks to reduce human annotation effort and improve scalability. Future work will aim to develop a more extensive and generalized set of action problem pairs, exploring such industrial applications as automated test case gen eration and proactive maintenance support for developers.