Study of whether AI detectors are more likely to flag essays written by non-native vs. native English speakers as AI

Background

As AI-generated writing has become increasingly common and convincing in schools and companies, detectors have been developed and marketed as a way to detect AI-generated content. However, the accuracy and reliability of these detectors are contested. This study analyses data about AI detectors and mainly focuses on whether there are differences in accuracy when judging essays written by non-native vs. native English speakers. It also studies whether detectors correctly identify AI or human-generated essays in general, and examines whether certain detectors are better than others.

This project is mainly a way for me, Michael Song, to explore and use modern R packages in the tidyverse, such as dplyr, ggplot2, readr, and purrr. It uses data from a 2023 issue Tidy Tuesday, a weekly data analysis community project.

You can check out the knitted analysis here!