What Is Deep Learning?

Deep learning, an advanced artificial intelligence technique, has become increasingly popular in the past few years, thanks to abundant data and increased computing power. It’s the main technology behind many of the applications we use every day, including online language translation, automated face-tagging in social media, smart replies in your email, and the new wave of generative models. While deep learning is not new, it has benefitted much from more availability of data and advances in computing.

ChatGPT, the AI-powered chatbot that has become the fastest growing app of all time(Opens in a new window), is powered by a deep-learning model that has been trained on billions of words gathered from the internet. DALL-E, Midjourney, and Stable Diffusion, AI systems that can generate images from text descriptions, are deep-learning systems that model the relation between images and text descriptions.

Deep Learning vs. Machine Learning

Deep learning is a subset of machine learning, a branch of artificial intelligence that configures computers to perform tasks through experience. Contrary to classic, rule-based AI systems, machine learning algorithms develop their behavior by processing annotated examples, a process called “training.”

For instance, to create a fraud-detection program, you would train a machine-learning algorithm with a list of bank transactions and their eventual outcome (legitimate or fraudulent). The machine-learning model examines the examples and develops a statistical representation of common characteristics between legitimate and fraudulent transactions.

After that, when you provide the algorithm with the data of a new bank transaction, it will classify it as legitimate or fraudulent based on the patterns it has gleaned from the training examples. As a rule of thumb, the more high-quality data you provide, the more accurate a machine-learning algorithm becomes at performing its tasks.

Machine learning is especially useful in solving problems where the rules are not well defined and can’t be coded into distinct commands. Different types of algorithms excel at different tasks.

Deep Learning and Neural Networks

While classic machine-learning algorithms solve many problems that rule-based programs have struggled with, they are poor at dealing with soft data such as images, video, sound files, and unstructured text.

For instance, creating a breast-cancer-prediction model using classic machine-learning approaches would require the efforts of dozens of domain experts, computer programmers, and mathematicians, according to AI researcher and data scientist Jeremy Howard in the above video.

The researchers would have to do a lot of feature engineering, an arduous process that programs the computer to find known patterns in X-ray and MRI scans. After that, the engineers use machine learning on top of the extracted features. Creating such an AI model takes years.

Artificial neural network
(Credit: Wikipedia)

Deep-learning algorithms solve the same problem using deep neural networks, a type of software architecture inspired by the human brain (though neural networks are different from biological neurons(Opens in a new window)). Neural networks are layers upon layers of variables that adjust themselves to the properties of the data they are trained on and become capable of doing tasks such as classifying images and converting speech to text.

Neural networks are especially good at independently finding common patterns in unstructured data. For example, when you train a deep neural network on images of different objects, it finds ways to extract features from those images. Each layer of the neural network detects specific features such as edges, corners, faces, eyeballs, and so on.

Top layers of neural networks detect general features. Deeper layers detect actual objects.
(Credit: arxiv.org)

Neural networks have existed since the 1950s (at least conceptually). But until recently, the AI community largely dismissed them because they required vast amounts of data and computing power. In the past few years, the availability and affordability of storage, data, and computing resources have pushed neural networks to the forefront of AI innovation.

Today, there are various types of deep-learning architectures, each suitable for different tasks. Convolutional neural networks (CNNs) are especially good at capturing patterns in images. Recurrent neural networks (RNNs) are good at processing sequential data such as voice, text, and musical notes. Graph neural networks (GNNs) can learn and predict relations between graph data, such as social networks and online purchases.

A deep-learning architecture that has become very popular recently is the transformer(Opens in a new window), used in large language models (LLMs) such as GPT-4 and ChatGPT. Transformers are especially good at language tasks, and they can be trained on very large amounts of raw text.

What Is Deep Learning Used For?

There are several domains where deep learning is helping computers tackle previously unsolvable problems:

Computer Vision

Computer vision is the science of using software to make sense of the content of images and video. This is one of the areas where deep learning has made a lot of progress. Beyond breast cancer, deep-learning image-processing algorithms can detect other types of cancer(Opens in a new window) and help diagnose other diseases(Opens in a new window).

But this type of deep learning is also ingrained in many of the applications you use every day. Apple’s Face ID uses computer vision to recognize your face, as does Google Photos for various features such as searching for objects and scenes as well as correcting images. Facebook used deep learning to automatically tag people in the photos you upload, before that feature was shut down in 2021.

Deep learning also helps social media companies automatically identify and block questionable content, such as violence and nudity. And finally, deep learning is playing a very important role in enabling self-driving cars to make sense of their surroundings.

Voice and Speech Recognition

When you speak a command to your Amazon Echo smart speaker or Google Assistant, deep-learning algorithms convert your voice to text commands. Several online applications also use deep learning to transcribe audio and video files. Google’s keyboard app, Gboard, uses deep learning to deliver on-device, real-time speech transcription that types as you speak.

Natural Language Processing and Generation

Natural language processing (NLP), the science of extracting the meaning of unstructured text, has been a historical pain point for classic software. Defining all the different nuances and hidden meanings of written language with computer rules is virtually impossible. But neural networks trained on large bodies of text can accurately perform many NLP tasks.

Google’s translation service saw a sudden boost in performance(Opens in a new window) when the company switched to deep learning. Smart speakers use deep-learning NLP to understand the various nuances of commands, such as the different ways you can ask for weather or directions.

Deep learning is also very efficient at generating meaningful text, also called natural language generation (NLG). Gmail’s Smart Reply and Smart Compose use deep learning to bring up relevant responses to your emails and suggestions to complete your sentences. A text-generation model developed by OpenAI created long excerpts of coherent text.

Large language models (LLMs) such as OpenAI’s ChatGPT can perform a wide range of tasks, including summarizing text, answering questions, writing articles, and generating software code. LLMs are being integrated in a wide range of applications, including corporate messaging and email apps, productivity apps, and search engines.

Art Generation

One field in which deep learning has become very useful recently is generating images. Models such as DALL-E and Stable Diffusion can create stunning images from textual descriptions. Microsoft is already using DALL-E in several products, including Designer. Adobe is also using generative models in several of its applications.

(Credit: fotograzia/Getty Images)

The Limits of Deep Learning

Despite all its benefits, deep learning also has some shortcomings.

Data Dependency

In general, deep learning algorithms require vast amounts of training data to perform their tasks accurately. Unfortunately, there’s not enough quality training data to create deep-learning models that can respond to many kinds of problems.

Explainability

Neural networks develop their behavior in extremely complicated ways—even their creators struggle to understand their actions. Lack of interpretability makes it extremely difficult to troubleshoot errors and fix mistakes in deep-learning algorithms.

Algorithmic Bias

Deep-learning algorithms are as good as the data they are trained on. The problem is that training data often contains hidden or evident biases, and the algorithms inherit these biases. For instance, a facial-recognition algorithm trained mostly on pictures of white people will perform less accurately for non-white people.

Lack of Generalization

Deep-learning algorithms are good at performing focused tasks but poor at generalizing their knowledge. Unlike humans, a deep-learning model trained to play StarCraft will not be able to play a similar game—say, WarCraft.

Also, deep learning is poor at handling data that deviates from its training examples, also known as “edge cases.” This can become dangerous in situations such as self-driving cars, where mistakes can have fatal consequences.

(Credit: Getty)

The Future of Deep Learning

In 2019, the pioneers of deep learning were awarded the Turing Award, the computer science equivalent of the Nobel Prize. But the work on deep learning and neural networks is far from over. Various efforts are in the works to improve deep learning.

Some interesting work includes deep-learning models that are explainable or open to interpretation, neural networks that can develop their behavior with less training data, and edge AI models, deep-learning algorithms that can perform their tasks without reliance on large cloud computing resource.

And although deep learning is currently the most advanced artificial intelligence technique, it is not the AI industry’s final destination. The evolution of deep learning and neural networks might give us totally new architectures.

This AI robot was made to be your best friend