Better AI Creative Writing?

Summary: In only a year, the creative writing abilities of ChatGPT have improved a little, as shown by a small experiment comparing the current model with output from mid-2023 in two genres: poetry and children’s stories. Bigger advances are expected next year.

OpenAI released an upgraded version of ChatGPT 4o on November 20, 2024, focusing on improved creative writing. In keeping with the tradition of horrible AI product names, the new model has the same name as the old one. There is no dot-release or other straightforward way of keeping track of the product releases so that we can refer to them in a simple manner.

The Abominable Snowman has retained his job as head of product naming strategy in OpenAI’s marketing department.

OpenAI releases new product versions without updating their names or version numbers. Worthy of The Abominable Snowman. (Midjourney)

The AI creator community has received this new release favorably, so I thought I would give it a go to see how much AI has improved its creative writing skills. I tried two experiments: Haikus and a children’s story. For both genres, I reused prompts from more than a year ago, so that we can compare the output.

In this article, I’m assessing the progress in AI’s creative writing skills. To give away the conclusion, it seems to have advanced a little, but not a lot, since last year. (Midjourney)

Case Study 1: Haikus

I put the new ChatGPT to the test to write Haikus about Jakob’s Law. Of the 8 I asked for, here are the three best:

Clicking where it shines,
Users trust patterns they know.
Don’t break their journey.

A grain of design,
Lost amid shifting currents.
Anchor with habit.

Years of clicking speak,
Habits carve paths in the mind.
Ignore them, they leave.

Compare with the Haiku I got from ChatGPT in September 2023:

Surfing other waves,
Expecting the familiar,
A web norm engraves.

AI trying its hand at being a Haiku master. Not much improvement over the last year. (Midjourney)

Instead of the strict 5-7-5 syllable count required of the Haiku poetic format, I asked for 3-line summaries of Jakob’s Law in a bullet point format. We could call this format “PowerPoint Poetry.”

🌐 Users learn from other sites.
📜 Patterns become expectations.
💔 Break them, and users leave.

🏝️ Your site is one among many.
🔗 Familiarity builds trust.
🚪 Confusion drives them away.

Again, compare with the result of the same exercise in September 2023:

🌍 The web is big. Your site is small.
🔄 Users have habits.
⚠️ Break habits, lose users.

Are the Haikus and PowerPoint Poetry from the upgraded ChatGPT in November 2024 better than the results from the old AI in September 2023? You tell me, I’m not sure. I do like the use of the desert island emoji with the solitary palm tree to symbolize a single website being a small isolated unit within a vast space, such as the Pacific Ocean.

For this exercise, I don’t feel that the GPT’s creative writing advances are as big as suggested by the hype. Marginally better? Maybe, but not significantly so.

Case Study 2: A Children’s Story

For my second experiment, I asked the new ChatGPT to write a children’s story about “the wildebeest who thought he was an impala.” I used this exact same prompt in August 2023 to generate an example for my article about AI creativity, so we have a base to compare with. The two stories are reproduced below.

A children’s story about the wildebeest who thought he was an impala. My second attempt to see whether AI has improved its creative writing since last year. (Midjourney)

Initially, ChatGPT gave me a story of 498 words, which is almost twice as much as the 265 words in last year’s story, making a fair comparison difficult. Luckily, accordion editing is well-supported by the recent addition of the “Canvas” feature in ChatGPT, so I easily had the story condensed to the 260-word version I’m showing you here.

(ChatGPT used different names for the animals this time, but I edited the new story to use the names from the first story to make the two versions easier to compare.)

The basic story is the same in both versions: the wildebeest tries to jump like an impala but can’t. Both versions even employ the word “thud” to describe how Wally lands. The stories’ morals are also the same: it’s more rewarding to be true to your actual nature than to pretend to be something else. (But that’s the moral from almost any American children’s story from the last 30 years. We have to look to stories from other countries, and preferably more than 50 years back, to find more nonconforming messages.)

However, I think the new story is better at setting up this conclusion, where the wildebeest realizes his special value to the happy community of savannah animals: a lion attacks the impalas, and Wally saves them. Even though this touch of danger might scare a very little child, it’s much more vivid than last year’s story, where Ina simply tells Wally that he can do things that impalas can’t. Show, don’t tell.

I also really like the following line from this year’s story:

"You saved us!" Ina said. Wally blinked. "I just did what wildebeests do." Ina smiled. "Exactly."

In conclusion, I do think ChatGPT improved its writing from August 2023 to November 2024 when it comes to the children’s story.

Conclusion: One Tie, One Win

This experiment was very small, but it indicated that ChatGPT has improved its creative writing prowess with this latest update. The poetry contest was probably a tie — at best, a marginal win — but the new version won for its children’s story.

As another example of the improvements in AI over the last year, I included the wildebeest illustration I ran in 2023 with my reprint of the 2023 manuscript below. Compared with the illustrations made by Midjourney now, the improvement is clear. Clearer than for the writing, I would say.

A wildebeest. Midjourney draws them better now than it did a year ago.

Overall, a little progress, but not a lot, over the last year. However, we should remember that we’re still within the same generation of AI models, even as they’re constantly being tweaked. Major steps forward can not be expected until completely new AI models are released, which is expected to happen in early 2025.

Cybercreativity is raining down on us. The downpour is expected to increase in 2025. (Midjourney)

Wildebeest Story from November 2024

In the golden grasslands of Africa, under a sky so blue it seemed painted, lived a young wildebeest named Wally. Unlike the other wildebeests, Wally gazed longingly at the impalas bounding gracefully nearby.

"Oh, to leap like that!" Wally sighed. "I must be an impala at heart!"

One day, Wally trotted up to a young impala named Ina. "Hello, fellow impala!" Wally said, puffing out his chest.

Ina tilted her head. "You’re a wildebeest."

"No, I’m one of you! Watch!" Wally crouched and launched himself into the air, managing a small hop before landing with a thud. Ina and the other impalas stifled giggles.

"You’re unique," Ina said kindly. "But being an impala isn’t just about leaping."

"Then teach me," Wally pleaded.

Ina shrugged and showed him how to leap, nibble leaves, and zigzag. But no matter how hard he tried, Wally’s hooves were too large, his legs too sturdy, and his body too heavy.

The impalas whispered, "Why not be the best wildebeest you can be?"

"I’m not special as a wildebeest," Wally said sadly.

One afternoon, a lion appeared. The impalas scattered, but Wally bellowed loudly, alerting his herd. Hundreds of wildebeests thundered across the plains, forcing the lion to retreat.

"You saved us!" Ina said. Wally blinked. "I just did what wildebeests do." Ina smiled. "Exactly."

Wally realized he didn’t need to be an impala to be special. He embraced being a wildebeest, sometimes joining the impalas’ games and laughing when he stumbled. Under the vast sky, Wally learned being yourself is the greatest leap of all.

(260 words)

Watercolor wildebeest, as drawn by Midjourney in November 2024.

Wildebeest Story from August 2023

Once upon a time, in the vast savannas of Africa, lived a young wildebeest named Wally. Wally was no ordinary wildebeest; he was different because he thought he was an impala.

Every morning, Wally would try to leap as high as the impalas, his dainty neighbors. He admired their sleek coats and swift speed. He even tried munching on leaves, just like them, even though he preferred the taste of grass.

One sunny day, Wally decided to join the impalas at the watering hole. They were leaping and frolicking around. Eagerly, Wally attempted a leap, but instead of a graceful jump, he stumbled and fell with a loud "thud."

The impalas stopped and looked at him, puzzled. "Wally, why are you trying to leap like us?" asked Ina, the lead impala.

Feeling shy, Wally confessed, "I want to be an impala like you because you're graceful and fast. I'm just a clumsy wildebeest."

Ina chuckled gently. "Wally, you have a strength that we impalas don't have. You are sturdy and resilient. You're part of a massive herd that moves together in unity, braving rivers filled with crocodiles. That's something we impalas can't do."

Wally paused and thought about Ina's words. He realized that he was special in his own way. Embracing his identity, Wally went back to his herd with a newfound confidence.

From that day forward, Wally took pride in being a wildebeest. He learned to love his uniqueness and understood that everyone has their own strengths. And that, little friends, is the tale of the wildebeest who once thought he was an impala.

(265 words)

Watercolor wildebeest, as drawn by Midjourney in August 2023.

About the Author

Jakob Nielsen, Ph.D., is a usability pioneer with 41 years experience in UX and the Founder of UX Tigers. He founded the discount usability movement for fast and cheap iterative design, including heuristic evaluation and the 10 usability heuristics. He formulated the eponymous Jakob’s Law of the Internet User Experience. Named “the king of usability” by Internet Magazine, “the guru of Web page usability” by The New York Times, and “the next best thing to a true time machine” by USA Today.

Previously, Dr. Nielsen was a Sun Microsystems Distinguished Engineer and a Member of Research Staff at Bell Communications Research, the branch of Bell Labs owned by the Regional Bell Operating Companies. He is the author of 8 books, including the best-selling Designing Web Usability: The Practice of Simplicity (published in 22 languages), the foundational Usability Engineering (27,796 citations in Google Scholar), and the pioneering Hypertext and Hypermedia (published two years before the Web launched).

Dr. Nielsen holds 79 United States patents, mainly on making the Internet easier to use. He received the Lifetime Achievement Award for Human–Computer Interaction Practice from ACM SIGCHI and was named a “Titan of Human Factors” by the Human Factors and Ergonomics Society.

· Subscribe to Jakob’s newsletter to get the full text of new articles emailed to you as soon as they are published.

· Follow Jakob on LinkedIn.

· Read: article about Jakob Nielsen’s career in UX

· Watch: Jakob Nielsen’s 41 years in UX (8 min. video)