Humans cannot tell when a poem was written by AI
People often praise poets for their soulful turns of phrase and how they can evoke deep emotions in a way that feels uniquely human. According to a recently published study in Scientific Reports, AI may be even more capable of touching our hearts. Not only do people struggle to tell the difference between AI-generated and human-written poetry, but many prefer AI-crafted poems to those produced by human effort, at least until they discover the silicon soul behind the words.
The University of Pittsburgh researchers tested how well readers could identify when a poem was written by OpenAI’s ChatGPT-3.5 AI model or by Shakespeare, Emily Dickinson, TS Eliot and others in the pantheon of English-language poetry. More than 1,600 participants read a random mix of ten poems, half by humans and half by the AI model. Many thought not only that humans wrote the AI poems, but that the human-written poems were the least likely to be flagged as such.
Apparently the complexity of human poetry was mistaken for confusing AI chatter. By avoiding the complexity often found in the work of classical poets, AI poetry can feel more relatable and less intimidating – qualities that readers unconsciously attribute to human creativity.
“We found that AI-generated poems were rated more favorably on qualities such as rhythm and beauty, and that this contributed to their misidentification as human-written poems,” the researchers wrote. “Our findings suggest that participants used shared but flawed heuristics to distinguish AI from human poetry: the simplicity of AI-generated poems may be easier for non-experts to understand, leading them to prefer AI-generated poetry and the complexity of misinterpreting human poems.” as incoherence generated by AI.”
Poetic AI
The inability of many people to tell when a poem was written by AI is surprising, as is the preference for poetry written by AI until the author is revealed. But rather it is a sign that poetry is not always easy to parse, especially if it is not contemporary. And it means that AI today tends to appeal to the largest possible number of people, just like its other products.
We often assume that human creativity is inherently superior or that we can intuitively recognize the work of a fellow human being. But as AI tools improve, these assumptions are increasingly challenged. This isn’t just an academic exercise either. It has real implications for art, education and the way we value creative work in a world where machines are now serious competitors.
The findings also suggest that as AI becomes an increasingly sophisticated creative tool, we may need to reconsider traditional definitions of artistry. It’s not necessarily about whether an AI can ‘feel’ or ‘imagine’, but how its output resonates with the audience.
But perhaps it is best to leave the last word on humanity and poetry to a poet who has written extensively about both. Here is “I am bound in a bundle of vain endeavors” by Henry David Thoreau:
I am tied to a package of vain efforts
By an accidental bond together,
Dangling this way and that, their links
Were made so loose and wide,
Thinks,
For milder weather.
A bunch of violets without roots,
And sorrel mixed,
Surrounded by a string of straw
Once coiled around their shoots,
The law
Which I’m stuck with.
A nosegay that grabbed Time from the outside
Those beautiful Elysian fields,
With weeds and broken stems, in haste,
Doth makes the mob flee
That garbage
The day he surrenders.
And here I bloom unseen for an hour,
Drink my juices,
Without root in the land
To keep my branches green,
But stand
In a bald cup.
Some soft buds were left on my stem
In imitation of life,
But oh! the children will not know,
Until time has withered them,
The woe
What they are full of.
But now I see that I was picked for a reason,
And then into the vase of life
Made of glass while I could survive,
But brought by a friendly hand
Alive
To a strange place.
That stock thus depleted will soon redeem its hours,
And by another year,
As God knows, with freer air,
More fruit and more beautiful flowers
Will wear,
As I drift away here.
