Peer-Reviews and AI Feedback Compared: University Students’ Preferences

Zapata, Gabriela C.; Saini, Akash K.; Cope, Bill; Kalantzis, Mary; Searsmith, Duane

Abstract

This article seeks to contribute to the existing literature on peer review by comparing human with machine reviews based on the implementation of a chatbot responding with reference to a large language model for the review of open-ended student work. Specifically, we sought to investigate how 91 graduate students in a public American university regarded both types of feedback after experiencing them on the same works. Data consisted of the participants’ written reflections comparing both review types after being exposed to them at least once, as well as the words they submitted to convey their views of peer comments. The thematic analysis of the data revealed a preference for human reviews, with students’ identifying a variety of academic and personal benefits. Despite this partiality, the reflections submitted by all participants also pointed to their acceptance of machine feedback as a welcomed complement to peer reviews, stating that the combination of both types of reviews had enriched their academic experience.

Citation

Zapata, G. C., Saini, A. K., Cope, B., Kalantzis, M., & Searsmith, D. Peer-Reviews and AI Feedback Compared: University Students’ Preferences

• Working Paper Type: Working Paper
• Deposit Date: Apr 8, 2024
• Publicly Available Date: Apr 9, 2024
• Public URL: https://nottingham-repository.worktribe.com/output/33294270
• Publisher URL: https://osf.io/preprints/edarxiv/uy8qp