Can AI actually write good fanfiction?

Since artificial intelligence-powered text-generation tools were made widely available to the public in the past few months, they’ve been heralded by some as the future of email, internet search, and content generation. But these AI-powered tools also have some clear shortcomings: They tend to be incorrect, and often generate answers that reinforce racial biases, for example. There are also serious ethical concerns about their unspecified training data.

It is not surprising that debates over using these tools have also been happening in fandom spaces. Excited fans almost immediately turned to them as a new way of exploring their favorite characters. With the right prompt, AI can spit out a few paragraphs of fic-like writing. But just as quickly, many fanfic writers began to speak out against the practice.

“Where is the AI sourcing information it is using? Other fanfictions, the actual original source, other stories? Would the AI be plagiarizing other fanfiction authors?” asks author Redd, who says that “overall, AI should be kept far away from fanfiction.”

The machine learning behind these AI tools typically requires them to be trained on large amounts of preexisting data, often scraped from the internet. But it’s hard to know exactly what gets caught in the crosshairs, or how much copyrighted writing is included. OpenAI, the company behind the GPT-4 system used as a basis for some of the popular text-generation tools currently available, has not revealed what data the model was trained on. (OpenAI did not respond to a request for comment on this topic in time for publication.) However, GPT-3 was trained on 45 terabytes of text data, much of which was taken from a wide web crawl.

Given the apparent extensiveness of this crawl, as well as these tools’ abilities to generate fic, many contributors to sites like Archive of Our Own and Fanfiction.net believe that the works published there may have been part of the data used to train these models. Some of these writers dislike that their work may have been used in such a way without their permission. On Archive of our Own, an increasing number of users are locking their works so that they can only be read by those logged into existing accounts. Users who lock these fics believe that it will help prevent their work from being found in AI data crawls.

Approached for comment, Archive of Our Own says it has “made some changes to the site code that will prevent responsibly coded bots from accessing it,” but that entirely restricting automated traffic isn’t an option as it would break the site’s interaction with search engines, screen readers, and other tools.

But these ethical questions are only part of the picture. Fanfic is also both an artistic and a community practice — and so far, AI tools can’t replicate either.

In my research for this article, I experimented with four different text generators: OpenAI’s direct access to ChatGPT, Bing AI, Google’s Bard, and Notion AI. I tested each out with the same broad prompts (“write me a fanfic”) at first, to see what they would come up with. Then, to recreate something closer to how they’re being used by some fans, I used more specific requests, including popular characters, ships, and tropes in fandom. Overall, it was an unrewarding experience. Each tool had its own approach to attempting fic, but the end results had broad similarities: The stories were generic and clumsy.

First, despite a common acceptance in fandom circles that these tools know what fic is, the tools often did not respond coherently to the broad prompt. At one point Bing AI produced a direct summary of the actual book Harry Potter and the Sorcerer’s Stone instead of any kind of fanfiction. When I asked what differences it would make to the original story, Bing AI instead searched for the differences between the Sorcerer’s Stone book and movie and told me several of them. When I said that’s not what I asked, it ended the conversation. When I prompted it again, more recently, it produced tips for writing fanfiction rather than generating the fic directly.

When these tools did generate something approaching fic, it wasn’t well written. Bard, for example, in response to the prompt “write me a fanfic,” put out a 377-word Harry Potter/Hermione Granger fic in which Harry helps Hermione with her homework. Anyone with a passing knowledge of the wizard franchise — so, almost everyone under 40 — would be able to recognize that this is out of character and that the roles should be swapped.

Halfway through the AI-generated fic, Harry and Hermione confess their feelings for each other and start kissing. This is not exactly an unusual occurrence in fanfiction, but there’s no emotion to the scene. It reads like a tween’s first attempt at understanding romantic writing. This is not an insult to tweens, who deserve to enjoy the creative process and practice their skills. There is plenty of fanfiction out in the world that reads similarly to Bard’s attempt — but there are also thousands that are much, much better.

Bard defaulting to Harry Potter fanfiction was not a one-off. Bing AI and OpenAI both did the same. (Notion AI did not appear to recognize what fanfiction was and produced stories that didn’t have any relation to existing media.) The stories that these AI tools created also focused on heterosexual couples. Testing the same basic prompt across these three tools multiple times each, they continued to return to straight Harry Potter ships: In addition to Hermione/Harry, they responded with Hermione/Ron, Hermione/Draco, and Luna/Draco, but never anything that wasn’t M/F.

Although many have left the Harry Potter fandom in recent years — as author J.K. Rowling has made a number of anti-trans statements — it is still a massive and active community, with the second-highest number of works on Archive of Our Own and almost 70,000 new works added in 2021. But the AI-generated works were not representative of the types of fic that real people actually write. On Archive of Our Own, Hermione/Draco is the most popular F/M ship, but even that is eclipsed by Harry/Draco, which has three times as many fics dedicated to it. It’s also much less popular than it once was. These days, “Marauders-era” fics — and often ones focusing on Remus/Sirius specifically — are a larger focus in the fandom.

And male-male relationships are almost always much more popular in fandom. As an artistic form focused on filling in the gaps left in existing media, adding queer relationships and validating queer readings of beloved characters is one of fanfic’s key motivators. AI tools, on the other hand, seemed to steer away from creating any M/M fic. Prompting each of the tools for Supernatural fanfic returned stories centered on Dean Winchester/Jo Harvelle and Sam Winchester/Jessica Moore, even though Dean/Castiel is far and away the most popular ship in any fandom on Archive of Our Own. There are more than 110,000 works in that AO3 tag alone, compared to only 811 Dean/Jo fics.

The AI algorithms clearly lean toward heterosexuality. That they also favor Harry Potter is related. Everything these generators produce leans generic, trending toward both the dominant model for sexuality and the media products with the largest overall cultural footprints. They do not — and cannot — easily respond to the nuances of fandom, with its much more diverse set of interests and a countercultural tendency toward queerness. In short, their responses to simple fic prompts are a good demonstration of how these tools fail to capture the human heart that sits at the center of any decent art or community.

More broadly, the habit of text generators to approach the average, rather than be able to adapt to the nuances of individual need, has much bigger ramifications than a fondness for straight ships. Tools trained on vast content scrapes will replicate whatever is most common. It’s not hard to imagine what perspectives get lost in that process.

Another major limitation is that these tools will not produce explicit material, as it violates some of their terms of service. OpenAI’s usage policies, for example, disallow “the description of sexual activity,” while Bing AI’s code of conduct prohibits creating or sharing “adult content.” But sex scenes, as in other kinds of literature, are an important part of fanfic. Of almost 11 million works on Archive of Our Own, almost 2 million, or around 18%, are rated Explicit. Another 15% are rated Mature. There’s a clear demand for this kind of writing that AI cannot currently fill.

The explicit content limitation can also get in the way of using the tools at all. Bing AI’s replies spool out a word at a time, so after I asked it for a (rating unspecified) Dean Winchester/Castiel fanfiction, I was able to see it write for a while before accidentally beginning to generate something explicit and having to reverse course. It suddenly deleted itself mid sentence and replaced it with “My mistake, I can’t give a response to that right now. Let’s try a different topic.” Repeating the prompt immediately afterward generated the response “I’m sorry but I cannot generate inappropriate content.” On a later test, however, the sudden deletion — after generating a few sentences of fic — happened again.

Image: Bing AI via Jay Castello

Bing also has the only UI that provides citations, with each message including “links to the web content it references.” Given more specific prompts, these citations linked out to very niche bits of content. For example, when asked for something in the hurt/comfort genre, its citations included author pages of specific writers on Archive of Our Own, as well as a Tumblr post with fewer than 200 notes. Microsoft’s developer blog doesn’t offer more explanation beyond calling these references citations, so it’s unclear whether these were scraped as part of the bot’s AI model or as part of the internet search the tool does to contextualize prompts. Regardless, it feels uncomfortable to be served these small creators’ works in a context that such creators likely did not imagine when publishing them.

But beyond all these limitations, there’s also a much more subjective, but fundamental issue with using these tools to generate fanfiction. Though they’ve been touted as tools that make creative exploration more accessible, using them to create fanfic simply didn’t feel creative or exploratory. I love writing. Teasing out interesting threads until I find enough that they can be pulled together into something narratively satisfying is one of my favorite things to do in the world. Writing fic is perhaps the most distilled version of that, because the threads are preexisting and I’m already invested in them. From my experience, using an AI tool does not recreate this feeling in the slightest. It doesn’t require or even encourage the reflective process of writing or the transformative work of fanfic, both of which can be tough but are deeply rewarding in and of themselves. Instead, there is nothing to enjoy except the finished product, which is typically flat and lifeless.

I’m not the only one who feels this way. “For me, the use of AI for fanfiction feels like it defeats the purpose,” says writer Seraph Abell. “The best part of writing is pouring yourself into your work and having people enjoy it. How can AI even recreate that kind of passion?” Fic is often deeply personal, too, which can’t be recreated by an AI. “You can see bits and pieces of me [in my fic] and you can see what I love about the show,” says Abell.

“When I write fanfiction, it is all coming from this place inside of me where I desperately need to get these words out about a piece of media that has touched my soul,” says Redd. “Writing fanfiction is about sharing and community, not just number of hits and bookmarks.”

In fact, most fic writers seem to feel this way. The “AI-generated” tag on Archive of Our Own is not well used, and many of the fics there are also tagged “crack” — i.e., the writer was just playing around. Many of them are explicitly poking fun at how bad the text generation is, or have simply described the generated text as an experiment. As a result, the site has not needed to make a blanket rule regarding AI-created stories, although it has fielded questions from users enough to have a preexisting response: “At the moment, we do not prohibit AI-generated fanworks as such, if they otherwise qualify as fanworks. However, we are monitoring the situation. Depending on the circumstances, AI-generated works could violate our anti-spam policies, but this is a developing situation.”

Fic writers have differing opinions on how AO3 and other sites could address AI works if they do become a bigger issue. “I’m not into censoring. So if someone wants to build their entire works list out of AI-generated work, go for it. Just tag it appropriately so everyone knows,” says Redd. Abell believes the works should be prohibited, predominantly to prevent any plagiarism of existing works, but similarly thinks that if they are allowed, they should be tagged so that readers can avoid them if they prefer.

For the moment, though, it seems like AI is more of a tool for fanfiction readers who want to consume something quickly and easily, perhaps for ship pairings or tastes that aren’t well covered by the fandom at large. These creations are for their own amusement, and they don’t seem to feel the need to share. Writers, on the other hand, are still writing. And that’s not likely to change anytime soon.