A Google AI project is smart enough to uncover real-world software vulnerabilities on its own, according to the company’s researchers.
Google’s AI program recently discovered a previously unknown and exploitable bug in SQLite, an open-source database engine. The company then reported the vulnerability before it reached an official software release, which prompted SQLite to issue a fix last month.
“We believe this is the first public example of an AI agent finding a previously unknown exploitable memory-safety issue in widely used real-world software,” Google’s security researchers wrote in a blog post on Friday.
The news joins growing research that shows today’s large language models hold the potential to find software vulnerabilities, potentially giving the tech industry a much-needed edge in defending software against hackers.
This isn’t the first time an AI program has discovered flaws in software. In August, for example, another large language model program called Atlantis uncovered a separate bug in SQLite. Meanwhile, machine learning models, a subset AI field, have been used for years to also find potential vulnerabilities in software code.
Still, Google says the achievement with its own AI program shows it’s possible for large language models to ferret out more complex bugs before the software itself is released. “We think that this is a promising path towards finally turning the tables and achieving an asymmetric advantage for defenders,” the company’s researchers wrote.
The Google project was originally dubbed “Project Naptime” before it became “Big Sleep,” a joke about how the company’s researchers are hoping the AI program will become capable enough to let Google’s human researchers “take regular naps” on the job.
Big Sleep was specifically designed with special tools meant “to mimic the workflow of a human security researcher” when examining the computer code of a specific program. Google also developed Big Sleep to look out for variants of existing security bugs, which are often a recurring problem in today’s software that hackers will eagerly exploit.
Recommended by Our Editors
“Recently, we decided to put our models and tooling to the test by running our first extensive, real-world variant analysis experiment on SQLite,” Google researchers wrote. This involved letting Big Sleep review recent changes made to SQLite’s code base. Google’s AI agent was able to investigate by triggering the bug and crashing SQLite to help it better understand and explain the problem through a root-cause analysis.
As a result, Google’s researchers wrote: “When provided with the right tools, current LLMs can perform vulnerability research.” That said, the blog post concedes that a specialized bug-finding tool known as “target-specific fuzzer,” which can inject random code into a program, would have also been effective in finding the same bug in SQLite.
Nevertheless, the company’s researchers conclude: “We hope that in the future this effort will lead to a significant advantage to defenders—with the potential not only to find crashing test cases, but also to provide high-quality root-cause analysis, triaging and fixing issues could be much cheaper and more effective in the future.”
Like What You’re Reading?
Sign up for SecurityWatch newsletter for our top privacy and security stories delivered right to your inbox.
This newsletter may contain advertising, deals, or affiliate links. Subscribing to a newsletter indicates your consent to our Terms of Use and Privacy Policy. You may unsubscribe from the newsletters at any time.
About Michael Kan
Senior Reporter
