Just over 20 years ago people didn’t even know what the internet was. Today we can’t even imagine our lives without it. Today I am going to give you a quick overview of what deep learning is and why it’s picking up right now.

And the reason why we are going back to the past is that neural networks along with deep learning have been around for quite some time and they’ve only started picking up now and impacting the world right now. But If you look back at the 80s you’ll see that even though they were invented in the 60s and 70s they really caught on to a trend or called the cold wind in the 80s so people are talking about them a lot. There was a lot of research in that area and everybody thought that deep learning or neural networks were these new things that are going to impact the world. That is going to change everything. That is going to solve all the world problems.

What happened? Why did the neural networks not survive and not change the world then? The reason for that is that they were just not good enough. They are not that good at predicting things and not that good at modeling.

Trending AI Articles:

1. How ethical is Artificial Intelligence?

2. Predicting buying behavior using Machine Learning

3. Understanding and building Generative Adversarial Networks(GANs)

4. AI & NLP Workshop

Or is there another reason?

Well actually there is another reason and the reason is in front of us. It’s the fact that technology back then was not up to the right standard to facilitate neural networks in order for neural networks and deep learning to work properly. You need two things:

1. data
2. strong computers to process that data

So let’s have a look at how data or storage of data has evolved over the years and then we’ll look at how technology has evolved.

Here we got three years 1956, 1980 and 2017.

How much did storage look back in 1956? Well, there’s a hard drive and that hard drive is only a 5GB. That’s five megabytes right there in the first picture and it is the size of a small room. In the first picture that’s a hard drive being transported to another location on a plane. And that is what storage looked like in 1956. In 1956 you had to pay two and a half thousand dollars of those days dollars to rent that hard drive to rent it not buy it, for one month.

In 1980 the situation improved a little bit. So here we got a 10-megabyte hard drive for three and a half thousand dollars. It is still very expensive and only 10 megabytes. So that’s like one photo these days. And today in 2018 we’ve got a 256 gigabyte SD card for $150 which can fit on your finger. And if you’re reading this blog a year later or like in 2020 or 2025 you probably laughing at us. All because by then you have even stronger storage capacity.

But nevertheless, the point stands. If we compare these across the board and we even taking price and size into consideration, so from 1956 to 1980 capacity increased about double. From 1980 to 2013 a huge jump in technological progress. And that stands to show that this is not a linear trend. This is an exponential growth in technology and If we add into account price and size you will be in the millions of increase.

And here we actually have a chart on a logarithmic scale.

If we plot the hard drive cost per gigabyte you’ll see that looks something like this. We’re very quickly approaching zero. Right now you can get storage on Dropbox and Google Drive which doesn’t cost you anything. Over the years this is going to go even further. Right now scientists are looking into using DNA for storage. And right now it’s quite expensive. It costs $7000 to synthesize two megabytes of data. But that kind of reminds you of this whole situation of the hard drive and the plan you know that this is going to be very very quickly. 20 years from now everybody’s going to be using DNA storage If we go down this direction. And here are some stats on that so you can explore it further. And basically you can store all of the world’s data in just one kilogram of DNA storage or you can store about 1 billion terabytes of data in one gram of DNA storage.

That’s just something to show how quickly we’re progressing and that this is why deep learning is picking up now. We are finally at the stage where we have enough data to train super cool and super sophisticated models. Back then in the 80s when it was first initially invented just wasn’t the case. And the second thing we talked about is processing capacity.

Here we’ve got an exponential curve again on a log scale. This is how computers have been evolving. This is called Moore’s Law, you’ve probably heard of it. You can see how quickly the processing capacity of computers has been evolving.

Right now we’re somewhere over here where an average computer can be bought for a thousand bucks at the speed of the brain of a rat. Between two and five years it will be the speed of a human or 20:23 and then by 2050 or 2045, it will surpass all of the humans combined. Basically, we’re entering the era of computers that are extremely powerful that can process things WAY faster then we can imagine. All of this brings us to the question: What is deep learning? and what is this whole neural network situation? What is going on? What are we even talking about here?

This gentleman over here Geoffrey Hinton is known as the godfather of deep learning. And he did research on deep learning in the 80s. He’s done lots and lots of research papers. He works at Google. So a lot of the things that we’re going to be talking about actually come from him and you can see a lot. He’s got quite a few YouTube videos. He explains things really well so I highly recommend checking them out.

And so the idea behind deep learning is to look at the human brain. This is going to be quite a bit of neuroscience coming up. And in these blog and ones coming up what we’re trying to do here is to see how the human brain operates.

You know we don’t know that much. You don’t know everything about the human brain but that little that we all know we want to mimic it and recreate it. And why is that? Well because the human brain seems to be one of the most powerful tools on this planet for learning, adapting skills and then applying them. If computers could copy that then we could just leverage what natural selection has already decided for us. All of that kind of algorithms that it has decided are the best which are going to leverage that. Why reinvent the bicycle ride? So let’s see how this works.

Here we’ve got some neurons so these neurons which have been smeared onto glass and then have been looked under a microscope with some coloring.

And this is you can see what they look like. They have a body, they have these branches and they have like tails and you can see that they have a nucleus inside in the middle. That’s basically what a neuron looks like in the human brain.

There are approximately 100 billion neurons all together so these are individual neurons. These are actually motor neurons because they’re bigger. They’re easier to see but nevertheless, there are a hundred billion neurons in the human brain. And it is connected to as many as about a thousand of its neighbors. So to give you a picture this is what it looks like. This is an actual data section of the human brain.

This is the cerebellum which is this part of your brain at the back. It is responsible for keeping a balance and some language capabilities and something like that. So this is just to show how works. How many neurons there are like billions and billions and billions of neurons all connecting. It’s like we’re talking about five or five hundred or a thousand or millions billions of neurons in there. And so that’s what we’re going to be trying to recreate. How do we recreate this on a computer? Well, we create an artificial structure called an artificial neural net where we have nodes or neurons and we’re going to have some neurons for input value so these are values that you know about a certain situation.

So, for instance, you’re modeling something you want to predict something you always could have some input something to start. Your prediction is off then that’s called the input layer. Then you have the output. So that’s of value that you want to predict or it’s surprising whether it’s is somebody going to leave the bank or stay in the bank. Is this a fraudulent transaction it’s a real transaction and so on. So that’s going to be the output layer. And in between, we’re going to have a hidden layer. So as you could see in your brain you have so many neurons. Some information is coming in through your eyes, ears, and nose so basically your senses.

And then it’s not just going right away to the output where you have the result. Is going through all of these billions and billions and billions of neurons before guess output. This is the whole concept behind it how we’re going to model the brain. We need these hidden layers that are there before the output to the input Layer neurons connected to hidden Layer neurons. And they connect to output Layer. This is pretty cool.

But what is this all about? Where is the deep learning here, or why is it called deep nothing deep in here? While this is kind of like an option which one might call shallow learning where there isn’t much indeed going on.

But why is it called deep learning Well because then we take this to the next level we separate it even further and we have not just one hit and there we have lots and lots and lots of hidden layers and then we connect everything just like in the human brain connect everything interconnected everything? And that’s how the input values are processed through all these hidden layers just like in the human brain.

Then we have an output value and now we’re talking deep learning.

So that’s what deep learning is all about on a very abstract level. And the further blogs I am going to write will dive deep into deep learning and by the end of it you will know what the deep learning is all about and you will know how to apply it in your projects.

Super excited about deep learning can’t wait to get started and I look forward to seeing you in the next blog or vlog.

Until then enjoy deep learning,

Manja

For more follow me on Linkedin, Instagram or Quora.