The 2023 launch of ChatGPT-4 marks the historic moment when AI chatbots jumped up to the level of global conversation. And in the relatively short span of time between then and now, AI writing tools such as Gemini, Copilot, DeepMind, Mistral, Claude Opus, and Sonnet have penetrated our everyday digital interactions, especially among students.
The technology is breathtaking. Yet, among techno-triumphalist proclamations of what machines can now do, a pervasive sense of unease lingers. Yes, AI chatbots can distill the essence of what has already been written. It can summarize. But can it also create? As humans create? Does it have the pulse of life in its words? Are these questions even in the right category? Or, is the very idea of a distinctly human creativity an obsolete normative peg around which the overarching narrative of AI invention refuses to be hung?
I had a chance to put these questions to the test at UC Santa Barbara’s Mellichamp Mind and Machine Center, a cross-disciplinary initiative created to understand the capabilities of the human mind and AI.Footnote 1 It was the spring of 2024 when the air on campus was thick with speculative and alarmist talk about what AI should and should not be allowed to do and when it can and cannot be used. In response, the Mind and Machine Center decided to stage a small experiment to find out what would happen if we officially allow AI in an open student competition. The theme of that year’s summit was “AI and Human Creativity,” and we launched an AI-inclusive contest in three categories, literature, music, and the visual arts, open to all UCSB undergraduate and graduate students, regardless of major.
In early 2024, this was not only unprecedented but also unsettling. And it caused a stir the moment it was announced. “Why present AI with an award for creative writing? What sad times,” one colleague wrote, a comment that, as it turned out, was on the gentler end of the spectrum in a debate that ensued across campus.
For some, the competition was seen as a form of technological cheerleading, a mindless surrender to the machines. While for others, myself included, this competition was a chance to see what happens when human minds and machine algorithm go head-to-head. It would be an opportunity to start tracking how this new technology changes how we write. Furthermore, it would provide a testing ground for examining whether AI shifts the meaning of writing itself.
Media theorists have long maintained that technological devices do not simply aid human communication; they change what communication means. Arguably Friedrich Kittler articulated this best. In Gramophone, Film, Typewriter (1999), he contended that the inventions of the phonograph, film, and typewriter around 1900 did not just improve how we record or transmit information; they rewired the relationship between people and meaning itself. It was, he suggested, the moment when humans became no longer the sole authors of what we said or saw, the moment when the tools we make started making us. As Kittler famously put it: “Media determine our situation.” In this light, I saw the AI-inclusive competition not as a provocation or a surrender but a way of making visible the changes created by the arrival of this new tool.
The competition rules were simple: create a new work, whether by AI, human/AI collaboration, or human imagination alone. All entries had to document the creation process in a separate file. Judges would review all submissions blindly, unaware of how each piece was made. The winning short story, artwork, and musical composition would each earn a $700 prize and a feature on the Mellichamp Mind and Machine website.Footnote 2
In what follows, I track my experience as Head Judge of the short story competition.Footnote 3 I record below the assumptions, biases, and expectations that influenced the judges as we approached the entries, and how these assumptions stood up against what we actually found. I also reflect on the kinds of creativity and blind spots we saw and some unexpected insights that were gained through this time-specific experiment, which turned out to be just as much about testing our own ideas of creativity as about choosing winners.
1. Preliminary questions
Before receiving the entries, the short story judges, Stephen Rachman, digital humanist and Americanist at Michigan State, Matthew Reynolds, Professor of English and novelist at Oxford, and I convened to attune expectations and set evaluative guidelines. It was notable how almost immediately and completely our discussion was framed by the “man versus machine” trope. Precedents such as the 1997 Kasparov vs. IBM Deep Blue chess match and the folklore of John Henry’s contest with the steam drill surfaced as cultural touchstones.
Acknowledging this genealogy of fear and fascination helped us articulate the central questions at stake. And they were: Will AI make human writers redundant? Will machine-generated literature be clunky and bot-like? Will we be able to tell which is which? Will AI inclusivity open the door to new creativity and new forms? And perhaps, most of all, Who will win? AI, human, or a hybrid of the two?
Alongside the hovering questions, there was also an unmistakable wave of promise. I was reminded of the 2016 AlphaGo match, in which DeepMind’s system defeated the legendary Go player Lee Sedol, an eighteen-time world champion, in a stunning 4–1 series. What had struck me most about AlphaGo at the time was not its extraordinary computational prowess but what the commentators at the time described as the “elegance” of its play. After the crushing defeat, Lee himself was not enraged but visibly moved, astonished, in particular, by what became known as “move thirty-seven,” a gesture which Lee characterized as “inhumanly creative.” The machine did not simply overpower the human; it produced a form of creativity that surprised even the most expert practitioners of the game. The hope, for us as judges, was that this blending of human and machine capacities might open the door to similar breakthroughs in literature, perhaps even yielding a “move thirty-seven” of literary form.
2. The competition
There were forty-one entries in all. We read them blind and made our notes independently. Once I began reading, the question of authorship, human or otherwise, slipped quickly to the background. What mattered during the reading process were the things that always matter: voice, style, the “pulse of life.” After we’d finished our individual assessments, we convened again to discuss our lists, the scores were added up and the winner was determined.
After selecting the winner, we learned that 16 entries were in the “human-only” category, 4 “machine-only,” and 21 were “human-machine collaboration.” To give you an idea of the entries, short sections from each category are shared below. And I invite you to share something of our experience of judging by taking a guess at whether the sections are written by humans, machines, or if it is a collaboration:
Sample One: From “Echoes of Steel and Flesh”
Sitting together on a fallen log, Mara and Eli watched as a gentle breeze stirred the branches ahead, sending a shower of golden leaves cascading down around them.
“It is beautiful here,” Mara whispered, her voice barely louder than a breath.
Sample Two: “From Being Ashore is a Trap”
Every creature of the sea boasts of being ashore, but my detective eye witness their fate sealed with ginger and green onions upon arrival. They are blanched, stir-fried, and consumed with voracity.
“Cute fish, you can’t escape.”
Sample Three: From “Stargazers”
“You’re only acting.” K told me.
K was correct. I do not possess emotions like humans do. It is a justifiable statement to say that whatever I feel remains a simulation, therefore an act.
The desert stretched out endlessly, vast and peaceful. The sand seemed to go on forever dunes rising and falling like an endless ocean frozen in time. Above, the stars shone brilliantly against the inky black sky, dots on a chalkboard. Sometimes the wind whipped up, blowing sand into my optical sensor lenses until my vision blurred, rendering me unable to accurately discern what lay ahead.
Uncomfortable.
Is it possible to draw a line between what is human-written, what is machine-written, and what is a hybrid of the two? This was a question that the judges raised before seeing the documentation file. As the short samples above suggest, distinguishing with any degree of certainty is exceedingly difficult. Sample Two “Being Ashore is a Trap” sounds a bit odd to the ear but is that evidence of AI? Or is the prose intentionally calling attention to the unnaturalness of the premise? All three judges felt certain before reading the documentation folder that Sample Two has high AI involvement. Yet it turned out that it was entirely human written. It was in fact it was the more “natural” sounding Sample One “Echoes of Steel and Flesh” which was entirely machine generated. The last sample “Stargazers” was a human/machine collaboration and the winner of the competition.
3. Reflections
The questions we asked before the competition were simple. The answers, on the other hand, were harder to come by, only after the judging was over did certain parts of those questions come into focus, illuminated not by theory but by what we actually read.
Will machine-generated literature be clunky and bot-like? Surprisingly, not as much as one may assume. In fact, it turns out that many humans write in ways that sound more mechanical and robotic than AI. One of the first surprises of the experiment was that the fully AI-generated stories did not fall to the bottom of the ranking. All four AI-only entries scored squarely in the middle range. That result, unremarkable as it may sound, has serious implications that go far beyond a short-story competition. Any kind of text, fictional or not, produced in 2 minutes by a large language model can easily earn a fair-to-middling grade, largely because AI is very good at the things our rubrics reward: standardized writing, tidy organization, descriptive passages, and realistic dialogue. Therefore, it raises a deeper question about how we assess prose. If academic essays prize clarity, structure, and grammatical competence, then AI writing which is typically fluent and conventional will meet the standard. That is why some argue that grading practices and assignments need to be redesigned, to expose what AI cannot do rather than reward what it does well. The challenge, of course, is how to do that—and this leads to the next finding.
Will we be able to tell which is which? The answer is a resounding no. The competition has made one thing abundantly clear: distinguishing between human- and machine-generated work is not merely difficult; it is virtually impossible. The line between the two has blurred to the point of vanishing, leaving us in a landscape where the meaning of authorship is changing. To return to Kittler’s media theory, he argued that once information moves from handwriting and print to technological media, humans no longer stand as the sole origin of meaning. We are now living inside that theory.
It is tempting to think the solution lies in new rules—tighter assignments, cleverer rubrics, tasks that privilege critical thought, interpretive nuance, and human originality. But such measures are ultimately unenforceable and maybe somewhat beside the point. What is needed is something more fundamental: a redefinition of what we mean by “good” writing. In the age of AI, these are not theoretical questions. They are practical questions on which the future of learning and perhaps of thinking itself will depend.
Will AI-inclusivity open the door to new creativity and new forms? The answer at this level of writing is for now, no. Or at least, not yet. Most of the stories we read followed a familiar structure: good versus evil, lessons learned, a clear resolution. But that pattern was not unique to the machine-generated pieces; the human submissions did much the same. There was nothing that felt like AlphaGo’s famous Move Thirty-Seven, the startling play that revealed a machine’s capacity for something resembling intuition. Then again, most of the entrants were novice writers, who had under a year of creative writing practice, and it may be that AI in the hands of professionals would produce different results.
That said, professional creative writers may also be the last group on earth to embrace AI. Daphne Ippolito’s study, “Creative Writing with an AI-Powered Writing Assistant: Perspectives from Professional Writers” shows that the technology has not reached a level that makes it genuinely useful for those already skilled in the craft of writing.Footnote 4 For now, the partnership between professional writers and AI remains a promise waiting to be fulfilled, partly because for many professional writers, AI simply slows down the process.
This brings me to the question of AI writing and time. As expected, the machine-only entries were produced with astonishing speed. Sarah Bacon, a novice creative writer, took less than an hour to generate Echoes of Steel and Flesh. It is likely to have taken far less than an hour but “under an hour” was the smallest unit of time available on the drop-down menu of the survey every participant completed. This was a critical oversight. We were so new to AI writing that it did not occur to us in 2024 to measure the time it takes to write a short story in minutes rather than hours.
What actually surprised us was that the human-only stories were not always the slowest to write. MingXin Du’s Being Ashore is a Trap took between 15 and 30 hours to complete without any AI assistance. But many of the human–AI collaborations, which we assumed would be quicker, turned out to be slower.
That finding unsettles one of the central myths about generative AI—that it is for time saving. In practice, the opposite can be true. Working with AI introduces its own kinds of cognitive labor: prompts to refine, mistakes to correct, an excess of options to sort through. What appears from the outside as acceleration can, in practice, be a different and curiously elaborate form of labor, one that also demand critical skills and judgment.
This leads me to Prompt Engineering skills. Another interesting discovery was the importance of developing what is now called “prompt engineering.” Many of the best human-AI collaborations relied heavily on the craft of designing precise, effective prompts. For instance, Rachel Chang, the winner of the competition, used Claude 3 Sonnet in a particularly savvy way, crafting prompts that guided the AI to produce exactly the kind of material she needed. Rachel’s process involved asking, “What would a robot (or AI) find interesting about stargazing?” She conceived the plot based on these answers and wrote the draft using a code-switching method, incorporating both traditional Chinese and English. She then used Claude 3 Sonnet to translate the entire text into English and followed up with her own revisions. She also used Claude 3 Sonnet to elaborate on scenic descriptions to match the atmosphere she wanted, and the direction specified in the prompt. Additionally, she asked Claude AI to provide descriptions of a character’s emotional shifts—transitioning from joy and excitement to disappointment and anger—maintaining a machine-like tone in line with the character’s robotic perspective.
This points to a new skill set for writers: not just writing but writing with AI, where a key part of writing is how well you can prompt the machine to align with your creative vision. The story of Rachel’s success was how Rachel and Claude 3 Sonnet translated and re-translated, revised and refined together in a subtle dance. It seems clear from this case study that the nature of human writing will not only change, but that it has also already changed.
Since the competition, I have been asking my students to document their use of AI in their essays. Almost all of them do, some in surprisingly ways. What I notice is the quality of work coming from the lower third of the class, which has improved markedly. I do not think this is due to a rise in completely machine-generated work. Technically, that is banned. And while the ban is impossible to enforce since I cannot detect AI writing with any real accuracy, the way the coursework is structured to scaffold writing development in weekly increments, there is little incentive to try. The improvement comes from editing. Students now have an extraordinary capacity to polish, reshape, and hone. Most essays are the product of an extended back-and-forth with AI, a dialogue not unlike Rachel’s collaboration with Claude.
Afterward, we learned that Rachel was taking a course called Critical A., when she took part in the competition, where she was taught prompt engineering. Guiding AI through prompts can take sophistication but it seems it can be taught. Her example suggests that “prompt literacy” has become part of what it now means to write well, a skill that sits alongside grammar and argument, and one that will only become more essential as the technology continues to evolve.
This brings me to a final question, one that crops up in every discussion of AI and writing: Will AI make human writing redundant? The competition offered an unexpected answer. Far from reducing the amount of human writing, AI looks set to increase it. The AI-inclusive format drew in a striking number of first-time writers, students who, without these tools, might never have tried to write a story at all, let alone participate in a literary competition. For me, that was the most heartening discovery. AI does not replace the impulse to write; in fact, AI activates writing.
This finding chimes with Ted Chiang’s essay “Why A.I. Isn’t Going to Make Art”, which argues that AI will not make art obsolete but will instead clarify what is distinctly human in creative labor.Footnote 5 Similarly, Jeanette Winterson, writing in The Guardian, suggests that AI’s very otherness offer what the human race needs.Footnote 6
I take something similar from the competition. Human writing fulfils a purpose that AI-generated text cannot replace but, paradoxically, helps to concretize. Many people feel compelled to write because if they do not, the inner flow of life they sense but do not understand and is not understood by society, would go unaddressed, unperceived, and unacknowledged. And the desire to break bridge that gap, break that silence, and make sense of things is a very deep human need. The power of writing to create a coherent reality for oneself, one’s culture, one’s history is not invalidated by AI-generated writing. If anything, it becomes more visible in contrast. The competition was not about machines outsmarting their makers, but about people finding new ways to speak. AI may change how we write but not why we write. The impulse remains the same: to shape the chaos of experience into something that can be seen, shared, and understood. Liberated from its instrumental function, I predict that human writing will be pushed toward a more expressive, symbolic mode than we have known before.Footnote 7
Author contribution
Investigation: S.P.
Open access
