No Humans-in-the-Loop: The People-less Stories Generated by GPT

In the last two years, numerous news outlets and academic articles have claimed that GPT-produced text is indistinguishable from human-written text across several genres. In this talk, Gabi Kirilloff and Claudia Carroll argue that not only does GPT not write fictional narratives like a human, but it also often fails to write about humans. Building on their previous work, they discuss how the distribution of stylistic features, including the frequency of verbs, nouns, and pronouns, can reliably distinguish human and GPT-written text. They argue that these textual markers point to a deeper conceptual difference between human and GPT-generated text—GPT-generated texts contain fewer characters and describe inanimate objects more often than people or characters. Kirilloff and Carroll close this talk by considering how these disembodied, perspectiveless narratives might affect narratives in the public sphere as LLMs become more widely used across sectors.

Modeling Culture is generously funded by an AI Lab Seed grant, with additional support from the Humanities Council and the Princeton Humanities Initiative.

Gabi Kirilloff is an Assistant Professor of English at Washington University in St. Louis, where she specializes in the intersection of the humanities and digital technology. Much of her work applies computational tools to study patterns and outliers in nineteenth and twentieth-century American fiction. She teaches and writes on a wide variety of topics, including data-analysis, video games, natural language processing, and women’s literature. She is the co-founder of the AI Humanities Lab at WashU. Her work has been published in journals including the Harvard Data Science Review, the Journal of Cultural Analytics, The Programming Historian, College Literature, and Digital Scholarship in the Humanities. She also writes about technology and culture for public venues, including Forbes.

Claudia Carroll is a Postdoctoral Research Associate at TRIADS. She received her PhD in English, specializing in characterization and computational literary studies, from the University of Notre Dame, where she was a Notebaert Premier Fellow. Her research uses quantitative methods to study characterization in fiction, with a focus on the nineteenth-century novel. At WashU, she is working on developing machine-learning methods for the analysis of reader cognition and on computational methods for the formal analysis of AI-generated literature. Her research is published or forthcoming in the Journal of Victorian Culture, Poetics Today, the Harvard Data Science Review, Digital Scholarship in the Humanities, and the Routledge Handbook of AI and Literature. For her work on narrative structure, she is the recipient of the 2022 Alan Nadal Prize from the International Society for the Study of Narrative. At WashU, she is a founding PI of the AI Humanities Lab and teaches in the Data Science and the Humanities (DASH) program.