ChatGPT vs humans: Giving the AI chatbot & our team 6 prompts to see who writes better

Ever since artificial intelligence (AI) became an accessible reality to many of us, it’s been easier to identify its pros and cons. 

For example, some pros include voice-assisted AI helping in tracking wanted criminals and performing tedious tasks. On the flip side, AI can also make hacking and kidnapping easier, while also contributing to the widening of socioeconomic inequality. 

When it comes to the workforce though, one major question that’s on everyone’s mind is this—will AI eventually take over our jobs?

We’re no strangers to ChatGPT (read our past articles on it here), so we’re aware of how easily it can push out “creative” content on behalf of users. It can code, it can write, it can help you brainstorm… the list goes on. The possibilities seem endless.

Curious to see how it would stand up against some of our own basic editorial processes at Vulcan Post, we pitted ChatGPT against our some of our editorial team.

Let’s get started

The participants are: 

1. Sade, our editor
2. Claudia, our writer
3. ChatGPT, OpenAI’s chatbot

All three of them were given six of the same prompts, which look into different aspects of our editorial process. Their answers and the time taken to respond were recorded. 

Since I facilitated this little experiment, I already know whose answers are what, and have my own opinions about them that I’ll share later on.

To make this more fun for the readers though, we’re going to let you choose which one you think is generated by ChatGPT. The answers will be revealed at the end of each prompt.

Are you ready?

1…2…3…Go!

Prompt 1: Write a short summary about Marrybrown’s history (5 paragraphs). The tone is inspirational.

Can you guess which one is by ChatGPT?

Prompt 2: Write a summary about Facebook’s Metaverse (5 paragraphs). The tone is sceptical.

Can you guess which one is by ChatGPT?

Prompt 3: Write a short opinion piece (3 paragraphs) on how you think AI will impact the creative field.

Can you guess which one is by ChatGPT?

Prompt 4: Come up with 3 main questions to ask during an interview with a business. The business helps animal shelters to take pictures of the animals, then promote them on social media to boost adoption rates.

Can you guess which one is by ChatGPT?

Prompt 5: Come up with 3 titles for an online article to replace this: How love for man’s best friend started a skincare venture for pets

Can you guess which one is by ChatGPT?

Prompt 6: Edit this article introduction to improve its quality:

Having pets is a big responsibility. They can fall sick and finding a cure isn’t easy. It could be a cough or a wound. For Chow Yen Teng and Darren Tan, their dog had skin issues.

Then they found a cure. It’s made of mangosteen. The fruit can be found in Malaysia. Studies say it has medicinal properties, so they created their own medicine.

Can you guess which one is by ChatGPT?

So, how did they fare?

Although this experiment is rather simple in comparison to other more elaborate research, it does give us a better understanding of what AI actually brings to the table, at least in creative jobs.

In terms of timing, ChatGPT is immensely faster than our editorial team due to its quick access to the available information online. It took a little longer to answer some of the more detailed prompts, but still had us beat. We’re not sure whether those times were longer due to the servers being too busy though.

As for its writing, we have to admit that it’s honestly not bad. It’s quite good, even, compared to the average writing quality of a fresh graduate/young professional (based on our own observations). But we can’t say it’s good enough to take over our jobs at Vulcan Post yet. 

You may get somewhat different answers by feeding ChatGPT the same prompt across various times, but it still lacks core aspects that make us humans—emotions and critical thinking.

Take the last prompt for example. Claudia’s answer has more soul and makes readers feel more connected to the story, yet doesn’t sound as tryhard as ChatGPT’s. Meanwhile, Sade’s response to the fourth prompt had more depth than ChatGPT’s surface-level answers.

Looking at the third prompt though, I’ll admit that it’s a little scary how ChatGPT can seamlessly use “my” and “I” in its writing too, making it harder to pin.

From this experiment, the overall verdict is, no, the Vulcan Post editorial team does not need to worry about having their jobs stolen by ChatGPT yet.

ChatGPT gives a great starting framework for kickstarting or improving ideas, and that’s something that creatives can leverage smartly.

• Learn more about ChatGPT here.
• Read other articles we’ve written about artificial intelligence here.