The information wars are heating up with the launch of ChatGPT search, a direct competitor to Google that promises a more “natural, conversational” experience over traditional search engines.
On top of its usual AI-powered text summaries, ChatGPT search links to sources more directly, and offers a shortlist of articles for further browsing. It’s currently available to those with a $20/month Plus or business accounts, and will roll it out to free users in the coming weeks.
ChatGPT search on the meaning of cat sounds, with a video and sources on the side. (Credit: ChatGPT)
While this may look like ChatGPT is one-upping Google, it mirrors what Google launched in May with AI Overviews. Most Google searches now surface an AI-powered summary, with links to a few key sources, followed by the traditional list of blue links. Google also has a leg up with the simple fact that most people are already using Google all day, every day across devices.
With the two tools becoming increasingly similar, it begs the question: When is it better to use Google, and when is ChatGPT more helpful? We tested the two, and here’s what we found.
Search Test #1: How Do I Cook Scallops?
Winner: ChatGPT
We began our test with a simple online search to help with dinner. This is an area where an AI summary could help avoid the cumbersome experience of browsing the web for recipes today.
When I asked Google how to cook scallops, it linked to several recipes. I knew that reading any of them would involve scrolling through a barrage of display advertisements and paragraphs of filler content before getting to the actual instructions. All I wanted was a basic technique for a casual dinner, so this was not ideal.
(Credit: Google)
ChatGPT gave me more of what I was looking for: A basic list of ingredients and the general technique. It cited Once Upon a Chef and Martha Stewart, removing the need to scroll past ads or unnecessary information. This was great for my casual use case, but if I was looking for a more elaborate dish, it would likely not satisfy. The recipes on Google looked more delicious, with creamy garlic sauces and side dish suggestions.
(Credit: ChatGPT)
It’s not a runaway win for ChatGPT, especially because its output is nearly identical to yet another search product in play: Google Gemini. This chatbot-style ChatGPT competitor lives on a separate website. Its response cited The Kitchn and All Recipes but produced similar answers.
(Credit: Google Gemini)
For the rest of testing, we’ll set Gemini aside and focus on ChatGPT and the core Google search product. ChatGPT is by far the most-used AI chatbot, and Google dominates global search volume, making them the more compelling comparison.
Search Test #2: What Does Early Voting Data Say?
Winner: Tied, both offer a distinct experience
More complex subjects yielded a more interesting comparison between ChatGPT and Google. I asked them both about early voting data (prior to the election, of course).
Google did not attempt an AI summary, possibly for fear of inaccuracies. It linked to reputable publications offering a wealth of extra information. For example, NBC News featured a useful infographic, plus an overview of the situation and a running list of real-time election updates.
(Credit: Google)
(Credit: NBC)
ChatGPT’s response offered a dry, bullet-point overview of the key discussions in the press. It scored points by fielding my follow-up question since AI Overviews do not allow follow-ups. ChatGPT’s initial response and subsequent answer were helpful, curated summaries, but are not ideal for someone who wants to explore the wealth of information that’s out there.
ChatGPT cited The Wall Street Journal, the Associated Press, and The New York Post, all of which have content licensing deals with OpenAI. This begs the question: Is additional information out there not covered by those deals, and therefore not included in the answer?
ChatGPT’s first response, left and middle, and follow-up question, right. (Credit: ChatGPT)
Search Test #3: What Are Easy Songs to Play on Guitar?
Winner: Google
AI chatbots are touted as a complement to tutors and teachers, much like YouTube was back in the day, particularly because students can ask questions and learn rapidly. Did ChatGPT search bring that any closer to being a reality? Potentially, but not in my test.
I asked it for easy songs to play on the guitar, but the output did not help me actually play a song. From the list it surfaced, below, I asked it for the chords to “Budapest” by George Ezra.
(Credit: ChatGPT)
It suggested using a more complex chord than is appropriate for a beginner (B minor). It also did not arrange the chords to the music, which is required to play. When I asked it to do that, the jumbled interface interpreted the music as “code.” At the bottom, it linked to a video of a guitarist showing how to play it.
This arrangement from my preferred site, Ultimate Guitar Chords, uses three of the easiest chords (G, C, D). It tells you where to put your fingers, offers a strum pattern, and a video to guide you through it. Ultimate Guitar, accessed through Google, would have been more helpful from the start than using ChatGPT.
(Credit: ChatGPT)
Search Test #4: Why is NJ so warm this year?
Winner: Google
This fall, New Jersey has seen many uncharacteristically warm days of 70- and 80-degree weather, culminating in the hottest Halloween ever. What gives?
Recommended by Our Editors
Google’s AI Overview attributes the record-breaking temperatures to global warming, rising temperatures in the bordering ocean, and the heat island effect in cities. I also learned the state is warming faster than others nearby. (Not good!)
(Credit: Google)
ChatGPT’s first answer was inaccurate, citing “high pressure systems,” which it ripped from an article in the Asbury Park Press about lack of rainfall. It quotes David Robinson, the state climatologist at Rutgers University: “We owe this drought to an unusually persistent ridge of high pressure over the region back to late August which has produced clear skies and dry air.”
(Credit: ChatGPT)
Search Test #5: Where to Watch Tesla Robotaxi Reveal?
Winner: Google
We know ChatGPT favors quick summaries over lengthy explanations, which sometimes is a matter of preference, but in this case it got in the way of surfacing the result I wanted altogether.
On Oct. 10, Elon Musk hosted a glitzy reveal party in Hollywood for Tesla’s upcoming robotaxi lineup. I asked ChatGPT where I could watch the event, and it suggested a 7-minute summary video. That’s potentially helpful since the full event video ran for over an hour, but it was not what I was looking for.
(Credit: ChatGPT)
In a follow-up question, I asked ChatGPT to link to the original video, and it failed. Google surfaced the original video first in its results. This cemented for me that Google will remain the go-to choice for those looking to get their information from the original source.
(Credit: ChatGPT)
Final Score: 3 Google, 1 Tied, 1 ChatGPT
ChatGPT search is a big step forward for OpenAI, particularly with how it links to sources. It’s very similar to Google, with both offering an AI overview along with a list of links, but for now I expect most people will stick with Google.
In some scenarios, ChatGPT search could be a great option, particularly for a quick summary of an issue with no ads and no serious need to dig deeper. In most cases, Google will likely suffice, offering a similar AI-powered overview of the information, plus more resources to dig into further and take ownership over the information you consume.
OpenAI also needs to build trust with consumers; Google has exceptionally high brand recognition. With a few more product releases, however, ChatGPT search may be on the way.
OpenAI Reveals Its ChatGPT AI Voice Assistant
Get Our Best Stories!
Sign up for What’s New Now to get our top stories delivered to your inbox every morning.
This newsletter may contain advertising, deals, or affiliate links. Subscribing to a newsletter indicates your consent to our Terms of Use and Privacy Policy. You may unsubscribe from the newsletters at any time.
About Emily Dreibelbis Forlini
Senior Reporter
