Funded Scholarship
Document Type
Article
Publication Date
2024
Publication Title
IEEE/ACM 21st International Conference on Mining Software Repositories
Abstract
Large Language Models (LLMs) have been widely adopted and are becoming ubiquitous and integral to software development. However, we have little knowledge as to how these tools are being used by software developers beyond anecdotal evidence and word-of-mouth reports. In this work, we present a study toward understanding how developers engage with and utilize LLMs by reporting the results of an empirical study identifying patterns in the conversation that developers have with LLMs. We identified a total of 19 topics describing the purpose of the developers in their conversations with LLMs. Our findings reveal that developers use LLMs to facilitate various aspects of their software development processes (e.g., information-seeking about programming languages and frameworks and soliciting high-level design recommendations) to a similar extent to which they use them for non-development purposes such as writing assistance, general purpose queries, and conducting Turing tests to assess the intrinsic capabilities of the models. This work not only sheds light on the diverse applications of LLMs in software development but also underscores their emerging role as critical tools in enhancing developer productivity and creativity as we move closer to widespread AI-assisted software development.
Recommended Citation
Mohamed, S., Parvin, A., & Parra, E. (2024, April). Chatting with AI: Deciphering Developer Conversations with ChatGPT. In 2024 IEEE/ACM 21st International Conference on Mining Software Repositories (MSR) (pp. 187-191). IEEE.
