Deep Learning – A Simple Explanation

In the recent years, Artificial Intelligence (AI) has been a subject of intense media speculation. Discussions related to machine learning, deep learning, and AI come up regularly in countless news articles, internet blogs and TV shows. We are being promised a future of intelligent robots, self-driving cars, transformative healthcare technology, and virtual assistants – a future sometimes painted in a grim light, where human jobs would be scarce and most economic activity would be largely handled by smart robots and better-than-human AI agents. Speculation or not, what is at stake here is our future, and it is important that we are able to recognise the signal in the noise, to tell apart world-changing developments from what are merely over-hyped media guesswork. To that end, this article is an attempt to provide a non-technical explanation of deep learning, an immensely rich and hugely successful sub-field of machine learning.

Since you are reading this article, I will assume that you have some understand of machine learning. If you don’t, I will highly recommend reading my earlier post on “Machine Learning – A Simple Explanation” before you continue with the rest of this article.

Deep learning is often presented by the media as shrouded in a certain mystery, with references to algorithms that “work like the human brain”, can “think” or “understand”. The reality is however quite far from this science-fiction dream. This analogy to the human brain is largely due to the use of a class of algorithms called “neural networks” in deep learning. The term “neural network” is a reference to neurobiology, a branch of biology that deals with the nervous system. Although some of the central concepts in deep learning were developed in part by drawing inspiration from our understanding of the brain, deep learning models are not models of the brain; and there is no scientific proof that the brain implements anything like the learning mechanisms in use in modern deep learning models.


Links to Relevant Books

So what is deep learning then? A concise definition of the field would be: the effort to automate intellectual tasks normally performed by humans. A slightly longer version would be: deep learning is a type of machine learning that trains a computer to perform human-like tasks, such as recognising speech, identifying images or making predictions. But the billion dollar question is: how do you train a computer to achieve such a feat?

To perhaps better appreciate the answer to this question, think about how we learn from experience, and how richer our experiences, the more we can learn. Also consider how a child learns to identify a cat among various animals by simply looking at many examples of cat images (or being pointed out repeatedly over time “cat, yes cat“), its behaviour and the differences between a cat and other animals. In deep learning, the same can be said for machines powered by hardware and software. The experiences through which machines can learn are defined by the data they acquire, and the quantity and quality of data determine how much they can learn. And similar to a child learning how to recognise a cat, deep learning algorithms learn by discovering intricate structures in the data they experience and building computational models to retain that knowledge (which gets progressively more meaningful with additional data).

For example, a deep learning model known as a convolutional neural network (CNN) can be trained to recognise objects using a large number (as in millions) of images, such as those containing cats. This type of neural network typically learns from the pixels contained in the images it acquires. During the training process, it learns to classify groups of pixels that are representative of a cat’s features, with groups of features such as claws, ears, and eyes indicating the presence of a cat in an image.

After machines have gained enough experience through deep learning, they can be put to work for specific tasks such as driving a car, detecting weeds in a field of crops, detecting diseases, inspecting machinery to identify faults, and so on.

Although deep learning is a fairly old subfield of machine learning, it only rose to prominence in the early 2010s (the reasons for which I will not go into in this article). In the few years since, it has achieved nothing short of a revolution in the field, with remarkable results in historically difficult areas of machine learning, such as “seeing” and “hearing” – problems which involve skills that seem very natural and intuitive to humans but have long been elusive for machines. In particular, deep learning has achieved the following breakthroughs:

1. Near-human level image classification.
2. Near-human level speech recognition.
3. Near-human level handwriting recognition.
4. Improved machine translation.
5. Improved text-to-speech conversion.
6. Digital assistants such as Alexa, Siri and Cortana.
7. Near-human level autonomous driving.
8. Improved text predictions.
9. Answering natural language questions.

But we are still just exploring the full extent of what deep learning can do, and given the size of investment that has been made in this area lately, we are very likely to see many more breakthroughs in the future! In the meantime, let me wrap up this article by quoting a few words of wisdom and caution from François Chollet, the creator of Keras, a leading deep learning framework for Python.

Although deep learning has led to remarkable achievements in recent years, expectations for what the field will be able to achieve in the next decade tend to run much higher than what will actually turn out to be possible. While some world-changing applications like autonomous cars are already within reach, many more are likely to remain elusive for a long time, such as believable dialogue systems, human-level machine translation across arbitrary languages, and human-level natural language understanding. In particular, talk of “human-level general intelligence” should not be taken too seriously. The risk with high expectations for the short term is that, as technology fails to deliver, research investment will dry up, slowing down progress for a long time.

Although we might have unrealistic short-term expectations for AI, the long-term picture is looking bright. We are only just getting started in applying deep learning to many important problems in which it could prove transformative, from medical diagnoses to digital assistants. While AI research has been moving forward amazingly fast in the past five years, in large part due to a wave of funding never seen before in the short history of A.I, so far relatively little of this progress has made its way into the products and processes that make up our world. Most of the research findings of deep learning are not yet applied, or at least not applied to the full range of problems that they can solve across all industries. Your doctor doesn’t yet use AI, your accountant doesn’t yet use AI. Yourself, you probably don’t use AI technologies in your day-to-day life. Of course, you can ask simple questions to your smartphone and get reasonable answers. You can get fairly useful product recommendations on Amazon.com. You can search for “birthday” on Google Photos and instantly find those pictures of your daughter’s birthday party from last month. That’s a far cry from where such technologies used to stand. But such tools are still just accessory to our daily lives. AI has yet to transition to become central to the way we work, think and live.

– François Chollet


Links to Relevant Books

Leave a Reply

Your email address will not be published. Required fields are marked *