In a surprising turn of events, a recent study conducted by researchers at the University of Pittsburgh has revealed that non-expert readers actually prefer AI-generated poetry over poetry created by humans. Not only that, but they are also more likely to mistake AI-generated poetry as human-authored, even when compared to poems written by famous English-language poets.
The study involved presenting participants with 50 poems written by 10 renowned English-language poets, alongside poems generated in the style of those poets by ChatGPT 3.5. It is worth noting that none of the poems chosen for each author were among their most well-known works, and both human and AI poems were limited to 30 lines or less.
The results of the study were quite surprising. The odds of a human-authored poem being identified as such were only around 75% when compared to an AI-generated poem being labeled as human-authored. Furthermore, participants ranked AI-generated poems higher in terms of overall quality than those written by humans.
This finding represents a significant shift from previous studies using earlier versions of the OpenAI platform, such as GPT-2. These earlier studies found that participants could reliably distinguish AI-generated poetry from human-authored poems.
The researchers behind the study suggest that the misidentification of AI-generated poems and the preference for them are linked. While ChatGPT 3.5 was trained to mimic the style of each poet, from Geoffrey Chaucer to Walt Whitman to Dorothea Lasky, the AI-generated poems lack the complexity and opacity of their human-authored counterparts. The accessibility of the more straightforward AI-generated poems appealed to non-expert readers, who then mistakenly interpreted this preference as evidence of human authorship.
It is understandable that non-experts may not derive the same level of enjoyment as literary buffs from the in-depth study and analysis of great human poetry. However, it is concerning to see the mismatch between their expectations and reality. As the study points out, these participants did not anticipate AI to be capable of producing poetry that they would enjoy at least as much as human-written poetry. This expectation, according to the study, is mistaken.
The study authors raise an important question: as generative AI models become more capable and prevalent, will ordinary people’s expectations for AI catch up to the reality? It is unclear at this point. The authors also emphasize the urgency of identifying effective disclosure methods, as simple disclosures of AI use are often overlooked.
This study sheds light on the evolving relationship between AI and human creativity. While AI-generated poetry may not match the complexities of human-authored works, it appears to have found a niche with non-expert readers who appreciate its accessibility. As AI continues to advance, it is crucial to examine our expectations and ensure effective communication between AI systems and human users. Only then can we truly appreciate the potential of AI in the realm of creativity.
Use the share button below if you liked it.