GPT-4 Offers Human-Level Performance, Hallucinations, and Better Bing Results

Yesterday, OpenAI announced the next iteration of its multimodal large language model called GPT-4(Opens in a new window), along with some impressive stats and a few familiar caveats.

When ChatGPT first appeared in November last year, it was like a light switch being turned on and we all gained access to a new level of artificial intelligence online. We’re all still getting used to that, but OpenAI hasn’t stood still. ChatGPT uses GPT-3.5, and OpenAI spent the last six months(Opens in a new window) using the lessons it learned from that alongside its adversarial testing program to create GPT-4, which blows GPT-3.5 out of the water.

OpenAI says GPT-4 is improved in most areas. It’s more creative and collaborative, accepts images as input, and can generate captions, classifications, and analyses from them. The new model can also handle over 25,000 words of text (8x more than ChatGPT) and surpasses ChatGPT in its advanced reasoning capabilities.

(Credit: OpenAI)

In terms of a performance comparison, GPT-4 outperforms GPT-3.5 across all types of exam, be that the Uniform Bar Exam, SATs, and various Olympiads. It offers human-level performance in these academic benchmarks, and achieves the 90th percentile in the Bar Exam, for example. GPT-4 also outperforms other large language models on all the traditional machine learning model benchmarks, even if you translate the questions into 24 different languages.

There are of course limitations, and OpenAI openly admits they are similar to those found in earlier versions of its language models. GPT-4 can and will “hallucinate” facts and make errors in reasoning. Social biases remain, and work is ongoing to better deal with adversarial prompts. It can be “confidently wrong in its predictions” and lacks knowledge on events that occurred after Sept. 2021 because that is its training data cut off point. With all than in mind, OpenAI’s advice is to not rely on GPT-4 “in high-stakes contexts.”

Microsoft has confirmed that Bing is running a version of GPT-4(Opens in a new window) which has been customized for search. Anyone using the new Bing preview has already experienced GPT-4 when searching, reading the answers it generates, or when having a chat. Microsoft also confirmed that any updates OpenAI makes to GPT-4 will be fed into Bing.

OpenAI has collaborated with other organizations to integrate GPT-4 into their products and services. The list includes Duolingo for its language-learning conversations, Stripe to help combat fraud, Morgan Stanley to help organize its knowledge base, the government of Iceland to help preserve its language, the Khan Academy as part of a learning pilot program, and Be My Eyes to aid visual accessibility.

Other than through using Bing, OpenAI is making GPT-4 accessible to ChatGPT Plus subscribers, but with a usage cap in place. There is also a waitlist for developers(Opens in a new window) who want access to the GPT-4 API, with new places being unlocked on a daily basis.

Mozilla Project Lead Talks App Privacy with PCMag