Science University Research Symposium (SURS)

Augmenting GitHub Issues using Apple App Store Reviews: A Replication Study

Publication Date

2026

College

College of Sciences & Mathematics

Department

Math and Computer Science, Department of

SURS Faculty Advisor

Esteban Parra Rodriguez

Presentation Type

Oral Presentation

Abstract

User reviews on app stores contain valuable insights regarding bugs and missing features, but the sheer volume makes it impractical for developers to read every single one. Pilone et al. (2025) proposed an automated approach that helps bridge this gap by linking reviews from the Google Play Store to relevant GitHub issues using semantic textual similarity. By converting both reviews and issues into numerical representations and finding the closest matches, developers can be automatically pointed to relevant user reviews. This paper presents a replication of their work by using the Apple App Store instead of Google Play reviews and investigating whether the method generalizes across platforms. We collect App Store reviews for 10 open source mobile applications with public GitHub repositories, using the same embedding model and pipeline as the original study, and compute similarity scores between reviews and issues. Our results show that the approach replicates for some apps but not others: Bitwarden achieves a mean similarity of 70.62% and 7.8% issue augmentation, nearly identical to the original paper's 70.5% and 8%, while six of the ten apps score about one standard deviation or more lower. Two annotators rated 218 matched pairs; pertinence rose with similarity (Spearman rho=0.31), but only 8.3% of pairs were strongly pertinent.

This document is currently not available here.

Share

COinS