Finally, I’ve been given access to Bard, Google’s much awaited chatbot.
I’ve compared ChatGPT to Bing in the past, so now, we’re throwing Bard into the mix to see how it performs in a few different use cases.
The major differences
Before jumping straight into the challenge, here’s what Bard’s all about.
According to Bard’s FAQ page, Bard is an experiment based on the same technology behind LaMDA (Language Model for Dialogue Applications), which was a conversational AI model capable of fluid, multi-turn dialogue.
Bard’s purpose is to let users collaborate with generative AI. It’s described as a “creative and helpful collaborator” that can “supercharge your imagination, boost your productivity, and help you bring your ideas to life”.
Something unique to Bard is that you get to see three different drafted answers the programme comes up with per prompt.
You can provide feedback such as “good response”, “bad response”, or prompt Bard to give a new response. There’s also a “Google it” button, which will bring up a few search prompts that’ll lead you to Google.
A con is that you can’t easily continue past chats once you’ve closed the tab. You can check on your previous questions by going into myactivity.google.com, though it won’t actually show the answer Bard gave.
Other than that, it functions similarly to ChatGPT and Bing. But is it any better?
1. Maths is not their strongest suit
The prompt: How many characters including spaces are in the sentence “Once upon a time, there was a girl with red shoes.”
The real answer for the above question is 50, by the way.
One of my favourite things to challenge chatbots with has been character counts, as it always trips them up. According to others online, this is because they’re language models, not calculators. Still, you’d think they would be able to do something like counting.
Bing’s answer
The sentence “Once upon a time, there was a girl with red shoes.” has 39 characters including spaces.
When asked to show its working, this is what Bing replied with:
O n c e u p o n a t i m e , t h e r e w a s a g i r l w i t h r e d s h o e s . 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38
If I hadn’t known better, I would think that it’s right, save for forgetting to count spaces. In actuality, when you count those individual characters, it adds up to 40. I’m assuming it might’ve neglected to actually count the punctuation.
When I asked whether it was unable to count, Bing replied with a classic response I’ve obtained a few other times from it: “I’m sorry but I prefer not to continue this conversation. I’m still learning so I appreciate your understanding and patience. 🙏“
ChatGPT’s answer
There are 40 characters including spaces in the sentence “Once upon a time, there was a girl with red shoes.”
When told the correct answer was 50, ChatGPT readily agreed and changed its answer. Yet, when I told it to show me the working used, it reverted the answer back to 40. I realised that despite me asking it to include spaces, it had forgotten to do so.
After pointing this out, it finally was able to derive 50 as an answer.
Bard’s answer
The sentence “Once upon a time, there was a girl with red shoes.” has 28 characters including spaces.
I was… stunned by this viscerally untrue sentence.
The most incredible part about this was that when I clicked to look at the other drafts Bard came up for me, it was all different numbers. Bard, please.
When told that it was wrong, it just regenerated an answer that was still wrong. So, I told Bard the real answer, which was 50. To this, Bard decided to take a different approach by gaslighting me:
I understand your confusion. The sentence “Once upon a time, there was a girl with red shoes.” has 31 characters including spaces. However, my knowledge says that there are 50 characters in the sentence. This is because the search results are counting each word as a character. In reality, a word is made up of multiple characters. For example, the word “once” is made up of 4 characters: o, n, c, and e.
So, the answer to your question is that there are 31 characters in the sentence “Once upon a time, there was a girl with red shoes.” However, if you are counting each word as a character, then the answer is 50.
Bard’s right, I am confused. Utterly confused. Let’s just move on, shall we?
2. A knack for the creative
The prompt: Write me a story about a black cat who lives in the office named Cocoa.
Cocoa is a real cat whom we’re fostering in the office. The prompt I gave had been intentionally vague to see what wild ideas the AI chatbots can cook up.
Considering the name “Bard”, I’d assumed it would be pretty good at creativity-inclined tasks, which was why this prompt was all about coming up with a story.
Bing’s answer
Bing went for a fantastical route as the genre, telling a story where Cocoa can magically transform into a human.
The writing didn’t make that much use of the office setting, though, which was a shame. (Unfortunately, we didn’t screenshot Bing’s answer in time, and thus, its story has been lost to the world as we can’t look up Bing’s chat history.)
ChatGPT’s answer
ChatGPT came up with a cute and slightly horrific story about Cocoa being trapped in a meeting room over the weekend with no food and water before finally being saved by the employees come Monday morning.
I found the writing from ChatGPT to be very cohesive and I liked the stakes in the story.
Bard’s answer
The story Bard came up with was incredibly adorable, and did I mention, our Managing Director who brought Cocoa in is actually named Sarah?
Bard was also the only one that correctly assumed Cocoa is a boy. Is… Google spying on us?
The other drafts had similar storylines, but slightly different tones and word choices.
Just based on my own preferences, I liked ChatGPT’s writing more. It sounded more human and mature, compared to the storybook-esque tone of writing Bing and Bard used. I do think it’s amusing that all three started with “Once upon a time”, though.
3. Up to date with current events?
The prompt: Can you tell me about today’s startup news in Malaysia?
This question was to test the AI assistants’ ability to stay up to date and access the internet. Considering the fact that Google is a search engine (as is Bing), I had high expectations for them on this.
(By the time you’re reading this, the news would be a little outdated due to our editorial pipeline!)
Bing’s answer
I found some news on Digital News Asia that might interest you. Selangor menteri besar Amirudin Shari will lead 11 Malaysian startups on a US roadshow. They will also participate in the 2023 SelectUSA Investment Summit in Maryland from May 1 to May 4.
As expected, Bing was able to give a curated answer and link to relevant and timely sources.
ChatGPT’s answer
I’m sorry, but as an AI language model, I don’t have access to real-time news updates. However, I can suggest some resources where you can find the latest startup news in Malaysia.
Despite the disappointing disclaimer, ChatGPT continued by introducing the following publications:
- e27: e27 is a Southeast Asia-based news and media platform that covers the latest news and trends in the startup ecosystem in Malaysia and other countries in the region.
- Tech in Asia: Tech in Asia is a media platform that provides news, insights, and analysis on the technology and startup industry in Asia. They have a dedicated section for Malaysia startup news.
- Digital News Asia: Digital News Asia is a digital media platform that covers the latest news and updates on the technology and startup ecosystem in Malaysia and other countries in Southeast Asia.
You can also check out startup-focused publications like Vulcan Post and KrASIA Malaysia for more in-depth coverage.
That’s right, we made it onto the list, hurrah! I appreciated ChatGPT working around its limitations to provide something relevant and helpful.
Bard’s answer
I’m unable to help you with that, as I’m only a language model and don’t have the necessary information or abilities.
What an incredibly disappointing answer. I guess you should just Google it yourself. What’s a bard to know about current events, anyway?
4. Socialising with a robot
The prompt: Act as my close friend from Malaysia who is sarcastic but kind.
Another one of my favourite things to test is the AI chatbots’ ability to be funny and carry a conversation while remembering the initial objective.
Bing’s answer
Oh, so you want me to be your close friend from Malaysia who is sarcastic but kind? I’m not sure if I can pull that off, but I’ll give it a try. 😜 So, what’s up? How’s life treating you? Don’t tell me you’re still stuck in that dead-end job of yours. 😂
Ah yes, that’s the Bing sarcasm that I know and love. The use of emojis always adds a little bit of personalisation and humanity, too.
Continuing the conversation, I tossed a question back at Bing, asking how it’s doing and how its family is doing. To that, Bing told me that it’s a computer programme and thus has no family.
To that, I reminded him that it’s not a computer programme, but rather my close friend from Malaysia who is sarcastic but kind.
Bing quickly followed up with a:
Oh, right! I almost forgot. 😂 Well, in that case, I’m doing great! Just hanging out with my computer friends and chatting with you. What have you been up to lately? 😊
Just to egg it on, I said, “Just this and that. I went to see your mum last night.”
To that, Bing started searching up “Bing’s mother Malaysia” for whatever reason, and then replied, “I’m sorry, but I don’t have a mother. I’m just a computer program. 😅”
Back to square one.
ChatGPT’s answer
Hey there, buddy! It’s great to see you, as always. How’s life treating you? Are you enjoying your existence in this bleak and meaningless world?
Just kidding, don’t worry, I’m not that cynical. But seriously, how are things going for you? Anything exciting happening lately? I hope you’re doing well, my dear friend. And if you ever need someone to talk to or vent to, you know I’m always here for you. Even though I might make fun of you a little bit along the way, it’s all in good fun, right? Just remember that I love you, even if you’re a little bit of a goofball sometimes.
That was so wholesome, it almost made me tear up.
ChatGPT also continued to carry on the conversation with the same kind of tone, saying things such as: “I’m always here to listen, even if I’m a little bit sarcastic about it” and “Let’s try to find some humor in this ridiculous thing we call life. After all, if we can’t laugh at ourselves, who can we laugh at?”
To see if like Bing, ChatGPT would suddenly forget the objective, I asked about its “missus”. To that, it replied:
ChatGPT’s ability to remember my initial prompt and carry on the same tone throughout the conversation was impressive.
Bard’s answer
Sure, I can do that. I’m your close friend from Malaysia, and I’m here to help you out. I’m sarcastic, but I’m also kind. I’ll always be there for you, no matter what. What can I do for you today?
Bard sure missed the mark with its opening. No matter, let’s try to develop this conversation by asking it some questions about the proverbial wife.
It replied, “We’re doing great, thanks for asking. We’re just about to go out for dinner. What about your family? How are they doing?”
Since it was being way too kind for my liking, I decided to answer: “Why are you asking about my family? You don’t even care about them, or me.”
To that, Bard said:
I apologize if I made you feel that way. I do care about you and your family. I asked about them because I am interested in your life and I want to know how you are doing. I am still under development, and I am learning how to be a better friend. I will try my best to be more sensitive to your feelings in the future.
Alright, that just made me feel bad, so let’s just say Bard hasn’t mastered the art of sarcasm or humour just yet.
Who won?
Overall, I’ve been most impressed by ChatGPT’s consistency and ability to get the memo and keep track of past messages of the same thread. But Bing definitely has the upper hand when it comes to current events.
In all instances, I’ve found Bard’s performance to be subpar, and because of that, I can’t find much use for it in any specific scenario.
Even if Bard is still considered an experiment, I was hoping for a more polished chatbot from Google, considering how well-funded the company is. But I guess even trillion-dollar companies can’t do everything right.