Hi, I'm Dan Mbaga with AWS AI. Welcome to the Introduction to Deep Learning Course. I have been with as for four years and I'm currently responsible for business development management in machine learning. As part of AWS AI Team, have worked with our customers to build their machine learning products from conception to production on AWS. In this video, you will learn about deep-learning or DL and about the services that AWS offers for developing deep-learning based applications. We'll also discuss a case study where one of our customers is innovating with Deep Learning. DL is a subset of machine learning, which itself is a subset of Artificial Intelligence. Scientists started studying deep-learning in 1950s and devoted significant resources to wait for the next 70 years. The foundation for the current error was laid in the 1980s and the 1990s with research from Yann LeCun on Convolutional Neural Networks, and along short-term memories or LSTM by Sepp Hochreiter and Juergen Schmidhuber. In 1986, the rediscovery of the backpropagation training algorithm marked a significant milestone in the study of Deep Learning. The backpropagation algorithm helps the model learn from his mistakes by leveraging the chain rule of derivatives. But in the decades that followed called a neural winter research into Deep Learning dropped off. This was partly due to limitations on data and compute. The introduction of the Internet, smartphones, smart TVs, and the availability of inexpensive digital cameras, meant that more and more data was available.. Computing power was on the rise. CPUs were becoming faster, and GPUs became a general-purpose computing tool. Both trends made neural networks progress. In 1998, Yann LeCun publish a paper on convolutional neural networks for image recognition tasks. But it wasn't until 2007 when the research began to accelerate again. The advent of GPU and reduction in training time, ushered in the mainstay of Neural Networks and Deep Learning. Both data and computing power made the task that neural networks tackled more and more interesting. With GPS becoming increasingly popular, Neural Networks resurface in 2008. Deep-learning uses Artificial Neural Networks to process and evaluate its reference data and come to a conclusion. Artificial Neural Networks or ANN are different from traditional Compute Processing Architectures. In that, they're designed to operate more like human brain. The more flexible and better at handling unanticipated anomalies, and novelty is in a data. We'll talk more about Artificial Neural Networks later. Deep Learning is a subset of machine learning algorithms. Deep-learning uses layers of non-linear Processing Units for features extraction and transformation. Each successive layer uses the output from the previous layer as an input. The algorithms may be supervised or unsupervised and applications include pattern analysis, which is unsupervised, and classification which could be supervised or unsupervised. These algorithms are also based on the unsupervised learning of multiple levels of features or representations of the data. Higher-level features are derived from low-level features to form a hierarchical representation. Deep learning algorithms are part of a broader Machine Learning field of learning representations of data, and they learn multiple levels of representations that correspond to different levels of abstraction. Where traditional Machine Learning focuses on feature engineering, Deep Learning focuses on end-to-end learning based on raw features. Each layer is responsible for analyzing additional complex features in the data. A Neural network is a collection of simple trainable mathematical unit that collectively learn complex functions. With enough training data, a Neural Network can perform a decent job of mapping input data and features to output decisions. It consists of multiple layers. There is an input layer, some hidden layers, and an output layer. The basic unit of an Artificial Neural Network is Artificial Neuron sometimes also called a node. Like the biological neurons for which they are named, artificial neurons have several input channels. A neuron sums these inputs inside of processing stage, and produces one output that can fan out to multiple other artificial neurons. In this simplified example, the input values are multiplied by the weight to get their weighted value. Then if appropriate, the node adds an offset vector to the sum called the bias, which adjusts to sum to generate more accurate predictions, based on the success or failure of the product predictions. Once the inputs have been weighted then sumed, and the bias is added if appropriate, the neuron activates if the final value produced by the preceding steps meets or exceeds the determined activation threshold. That's called the activation step. It's the final step before an output is deliver. The Feedforward Neural Network is any neural network that doesn't form a cycle between neurons. This means data moves from one input to output without looping backward. This another Neural Network that does look backwards, and that's called Recurrent Neural Network. The primary value of a recurrent neural network comes when processing sequential information such as text, or speech, or handwriting. Where the ability to predict the next word or later is vastly improve if you're factoring in the words, or letters that come before it. Recurrent Neural Networks became much more popular after 2007 when Long Short Term Memory or LSTM approaches revolutionized speech recognition programs. LSTM is now the basis for many of today's most successful applications in the speech recognition domain, text-to-speech domain, and handwriting recognition domain. Use cases for Deep Learning span multiple industries. Let's have a look at each. You can find text analysis use cases in a finance, social, CRM, and insurance domains to name a few. It's used to detect insider training. Check for regulatory compliance. Brand affinity. Sentiment analysis. Intent Analysis, and more by essentially analyzing blobs of text. Deep Learning is also used to solve problems around time-series and predictive analysis. It's using datacenters for Log Analysis and risk fraud detection, by the supply chain industry for resource planning, and in the IoT field for predictive analysis using sensor data. It's also used in social media, and e-commerce for building recommendation engines. It's using sound analysis too. You find Deep Learning being used in the security domain for voice recognition, voice analysis, and in the CRM domain for sentiment analysis. You'll also find deep learning in both the automotive and aviation industries, where it's used for Engine and instrument floor detection. You'll even find deep learning in the finance industry for credit card fraud detection among other things. Finally, it's used for image analysis. In the security domain, Deep Learning is used for things like facial recognition. In social media, it's used for tagging and identifying people in pictures.. The challenge of course is scale. In 2012, AlexNet won the convolutional neural network ImageNet competition. It consisted of eight layers, 650,000 interconnected neurons, and almost 60 million parameters. Today, the complexity of Neural Networks has increased significantly. With recent networks such as Resnet 152, a Deep Residual Neural Network which has a 152 layers, and millions more connected neurons in parameters. The AWS Platform offers three advanced Deep Learning enabled managed API services. Amazon Lex, Amazon Polly, and Amazon Rekognition. Amazon Lex is a service for building conversational interfaces into any application using voice and text. It provides that advanced Deep Learning functionalities of automatic speech recognition, for converting speech-to-text, and natural language understanding to recognize the intent of the input. That enables you to build applications with highly engaging user experiences, and life-like conversational interactions. Amazon Polly turns tags into lifelike speech. Allowing you to create applications that talk, and build entirely new categories of speech enabled products. Amazon Rekognition makes it easy to add image analysis to your applications, so that your application can detect objects, scenes, and faces, and images. You can also search and compare faces, recognize celebrities, and identify inappropriate content. Deep Learning can often be technically challenging. Requiring you to understand the math of the models themselves, and the experience in skating, training, and inference across large distributed systems. As a result, several Deep Learning frameworks have emerged, which allow you to define models and then train them at scale. You can build custom models using the Amazon deep-learning AMIs. Built for Amazon Linux and Ubuntu. The AWS Deep Learning AMIs come pre-configured with Apache MXnet TensorFlow, the Microsoft Cognitive Toolkit Caffe, Caffe2, theano, torch, Pytorch and Keras. The Deep Learning AMIs enable you to quickly deploy and run any of these frameworks at scale. The Deep Learning AMIs can help you get started quickly. They're provisioned with many deep learning frameworks including tutorials that demonstrate proper installation, configuration, and model accuracy. The Deep Learning AMIs install dependencies, track library versions, and validate code compatibility. With updates to the AMIs every month, you always have the latest versions of the engines in data science libraries. Whether you need Amazon EC2 GPU or CPU Instances. There's no additional charge for the deep learning AMIs. You only pay for the AWS resources that you need to store and run your applications. There are two ways to get started with AWS Deep Learning AMIs; you can deploy a deep-learning Compute Instance in one click. The AWS Deep Learning AMIs can quickly be launched from AWS marketplace. You have the choice of GPUs for large-scale training, and CPUs for running predictions or inferences. Both of them give you a stable, secured, and high-performance execution environment to run your applications with pay-as-you-go pricing model. The other way to get started with the Deep Learning AMIs is to launch an AWS Cloud Formation Deep Learning template. To train over multiple instances, you use the Deep Learning CloudFormation template for a simple way to launch all of your resources quickly using the deep learning AMIs. Now, let's talk about a use case. C-Span is non-for-profits surveys focused on broadcasting and archiving US government proceedings. C-span had developed an automated facial recognition solution to help human indexers, but it will slow. They could only index half of the incoming content by speaker limiting the ability of users to search archive content. So what are the solutions and benefit that they had using Amazon Rekognition as a Deep Learning Service? They implemented Amazon recognition to automatically match, uploaded screenshots of a collection of 97,000 known faces. That enables C-span to more than double the video index from $3,500-7,500 per year. It drove down to labor required to index an hour of video content from 60 minutes to 20 minutes. They deployed it in less than three weeks, and index at 97,000 image collection in less than two hours. I hope you learned a little something, and we'll continue to explore other courses. I'm Dan Mbaga with AWS AI, and thanks for watching.
