No, That’s Wrong: Google’s Bard AI Demo Spouts Incorrect Info

As Google’s AI-powered Bard prepares to compete against ChatGPT, don’t count on the chatbot programs always being right: A recent demo of Bard shows it spouting inaccurate information. 

Bard, which Google announced on Monday, is slated to arrive in the coming weeks. To promote the AI program, the company posted a GIF(Opens in a new window) on social media that shows Bard answering a question about what new discoveries NASA’s James Webb Space Telescope has made.

The program lists three discoveries the space telescope made in an easy-to-read, bulleted format. Hence, through Bard, a user can quickly learn information, without having to scroll through a long list of search results to find the applicable site. 

(Credit: Google)

The only problem? Astronomers were quick to point out(Opens in a new window) one of the listed facts that Bard generated is wrong. The program erroneously claims the James Webb Space Telescope “took the very first pictures of a planet outside of our own solar system.” But in reality, astronomers snapped the first image of an exoplanet(Opens in a new window) in 2004 using the European Southern Observatory’s Very Large Telescope in Chile.

Bard may have made the mistake because the James Webb Space Telescope did take its first image of a planet outside of our own solar system in September—it just wasn’t the first telescope to ever do so. NASA’s “ambiguous” press releases on the exoplanet discoveries from James Webb may have further confused the AI program, added(Opens in a new window) astronomer Bruce Macintosh.

Tweet(Opens in a new window)

The mistake is a bit embarrassing since Bard is intended to enhance the search experience, not act as a detriment. In a statement, the company emphasizes that Bard is still a work-in-progress. 

“This highlights the importance of a rigorous testing process, something that we’re kicking off this week with our Trusted Tester program,” the company told PCMag. “We’ll combine external feedback with our own internal testing to make sure Bard’s responses meet a high bar for quality, safety, and groundedness in real-world information.”

Nevertheless, the error shows a risk of using Bard or ChatGPT to streamline search results. The same programs can accidentally serve up misinformation in trying to sum up content. Microsoft, which is integrating ChatGPT into the Bing search engine, even concedes: “Bing will sometimes misrepresent the information it finds, and you may see responses that sound convincing but are incomplete, inaccurate, or inappropriate.”

Microsoft is trying to address the misinformation risk by requiring the new ChatGPT-powered Bing to cite sources and show how it arrived at an answer. Still, the approach assumes the user will take the time to check sources when many might not. 

Both the new AI-powered Bing and Bard have yet to fully launch. But when they do, their capacity to post misinformation and be exploited for abuse will no doubt be tested. Microsoft’s own FAQ(Opens in a new window) on the ChatGPT-powered Bing adds a disclaimer: “Use your own judgment and double check the facts before making decisions or taking action based on Bing’s responses.”