Deep Learning Essentials电子书

Title Page

Copyright and Credits

Deep Learning Essentials

Packt Upsell

Why subscribe?

PacktPub.com

Contributors

About the authors

About the reviewer

Packt is searching for authors like you

Preface

Who this book is for

What this book covers

To get the most out of this book

Download the example code files

Download the color images

Conventions used

Get in touch

Reviews

Why Deep Learning?

What is AI and deep learning?

The history and rise of deep learning

Why deep learning?

Advantages over traditional shallow methods

Impact of deep learning

The motivation of deep architecture

The neural viewpoint

The representation viewpoint

Distributed feature representation

Hierarchical feature representation

Applications

Lucrative applications

Success stories

Deep learning for business

Future potential and challenges

Summary

Getting Yourself Ready for Deep Learning

Basics of linear algebra

Data representation

Data operations

Matrix properties

Deep learning with GPU

Deep learning hardware guide

CPU cores

CPU cache size

RAM size

Hard drive

Cooling systems

Deep learning software frameworks

TensorFlow – a deep learning library

Caffe

MXNet

Torch

Theano

Microsoft Cognitive Toolkit

Keras

Framework comparison

Setting up deep learning on AWS

Setup from scratch

Setup using Docker

Summary

Getting Started with Neural Networks

Multilayer perceptrons

The input layer

The output layer

Hidden layers

Activation functions

Sigmoid or logistic function

Tanh or hyperbolic tangent function

ReLU

Leaky ReLU and maxout

Softmax

Choosing the right activation function

How a network learns

Weight initialization

Forward propagation

Backpropagation

Calculating errors

Backpropagation

Updating the network

Automatic differentiation

Vanishing and exploding gradients

Optimization algorithms

Regularization

Deep learning models

Convolutional Neural Networks

Convolution

Pooling/subsampling

Fully connected layer

Overall

Restricted Boltzmann Machines

Energy function

Encoding and decoding

Contrastive divergence (CD-k)

Stacked/continuous RBM

RBM versus Boltzmann Machines

Recurrent neural networks (RNN/LSTM)

Cells in RNN and unrolling

Backpropagation through time

Vanishing gradient and LTSM

Cells and gates in LTSM

Step 1 – The forget gate

Step 2 – Updating memory/cell state

Step 3 – The output gate

Practical examples

TensorFlow setup and key concepts

Handwritten digits recognition

Summary

Deep Learning in Computer Vision

Origins of CNNs

Convolutional Neural Networks

Data transformations

Input preprocessing

Data augmentation

Network layers

Convolution layer

Pooling or subsampling layer

Fully connected or dense layer

Network initialization

Regularization

Loss functions

Model visualization

Handwritten digit classification example

Fine-tuning CNNs

Popular CNN architectures

AlexNet

Visual Geometry Group

GoogLeNet

ResNet

Summary

NLP - Vector Representation

Traditional NLP

Bag of words

Weighting the terms tf-idf

Deep learning NLP

Motivation and distributed representation

Word embeddings

Idea of word embeddings

Advantages of distributed representation

Problems of distributed representation

Commonly used pre-trained word embeddings

Word2Vec

Basic idea of Word2Vec

The word windows

Generating training data

Negative sampling

Hierarchical softmax

Other hyperparameters

Skip-Gram model

The input layer

The hidden layer

The output layer

The loss function

Continuous Bag-of-Words model

Training a Word2Vec using TensorFlow

Using existing pre-trained Word2Vec embeddings

Word2Vec from Google News

Using the pre-trained Word2Vec embeddings

Understanding GloVe

FastText

Applications

Example use cases

Fine-tuning

Summary

Advanced Natural Language Processing

Deep learning for text

Limitations of neural networks

Recurrent neural networks

RNN architectures

Basic RNN model

Training RNN is tough

Long short-term memory network

LSTM implementation with tensorflow

Applications

Language modeling

Sequence tagging

Machine translation

Seq2Seq inference

Chatbots

Summary

Multimodality

What is multimodality learning?

Challenges of multimodality learning

Representation

Translation

Alignment

Fusion

Co-learning

Image captioning

Show and tell

Encoder

Decoder

Training

Testing/inference

Beam Search

Other types of approaches

Datasets

Evaluation

BLEU

ROUGE

METEOR

CIDEr

SPICE

Rank position

Attention models

Attention in NLP

Attention in computer vision

The difference between hard attention and soft attention

Visual question answering

Multi-source based self-driving

Summary

Deep Reinforcement Learning

What is reinforcement learning (RL)?

Problem setup

Value learning-based algorithms

Policy search-based algorithms

Actor-critic-based algorithms

Deep reinforcement learning

Deep Q-network (DQN)

Experience replay

Target network

Reward clipping

Double-DQN

Prioritized experience delay

Dueling DQN

Implementing reinforcement learning

Simple reinforcement learning example

Reinforcement learning with Q-learning example

Summary

Deep Learning Hacks

Massaging your data

Data cleaning

Data augmentation

Data normalization

Tricks in training

Weight initialization

All-zero

Random initialization

ReLU initialization

Xavier initialization

Optimization

Learning rate

Mini-batch

Clip gradients

Choosing the loss function

Multi-class classification

Multi-class multi-label classification

Regression

Others

Preventing overfitting

Batch normalization

Dropout

Early stopping

Fine-tuning

Fine-tuning

When to use fine-tuning

When not to use fine-tuning

Tricks and techniques

List of pre-trained models

Model compression

Summary

Deep Learning Trends

Recent models for deep learning

Generative Adversarial Networks

Capsule networks

Novel applications

Genomics

Predictive medicine

Clinical imaging

Lip reading

Visual reasoning

Code synthesis

Summary

Other Books You May Enjoy

Leave a review – let other readers know what you think