Deep Learning: Practical Neural Networks with Java电子书

Deep Learning: Practical Neural Networks with Java

Table of Contents

Deep Learning: Practical Neural Networks with Java

Deep Learning: Practical Neural Networks with Java

Credits

Preface

What this learning path covers

What you need for this learning path

Who this learning path is for

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Java Deep Learning Essentials

1. Deep Learning Overview

Transition of AI

Definition of AI

AI booms in the past

Machine learning evolves

What even machine learning cannot do

Things dividing a machine and human

AI and deep learning

Summary

2. Algorithms for Machine Learning – Preparing for Deep Learning

Getting started

The need for training in machine learning

Supervised and unsupervised learning

Support Vector Machine (SVM)

Hidden Markov Model (HMM)

Neural networks

Logistic regression

Reinforcement learning

Machine learning application flow

Theories and algorithms of neural networks

Perceptrons (single-layer neural networks)

Logistic regression

Multi-class logistic regression

Multi-layer perceptrons (multi-layer neural networks)

Summary

3. Deep Belief Nets and Stacked Denoising Autoencoders

Neural networks fall

Neural networks' revenge

Deep learning's evolution – what was the breakthrough?

Deep learning with pre-training

Deep learning algorithms

Restricted Boltzmann machines

Deep Belief Nets (DBNs)

Denoising Autoencoders

Stacked Denoising Autoencoders (SDA)

Summary

4. Dropout and Convolutional Neural Networks

Deep learning algorithms without pre-training

Dropout

Convolutional neural networks

Convolution

Pooling

Equations and implementations

Summary

5. Exploring Java Deep Learning Libraries – DL4J, ND4J, and More

Implementing from scratch versus a library/framework

Introducing DL4J and ND4J

Implementations with ND4J

Implementations with DL4J

Setup

Build

DBNIrisExample.java

CSVExample.java

CNNMnistExample.java/LenetMnistExample.java

Learning rate optimization

Summary

6. Approaches to Practical Applications – Recurrent Neural Networks and More

Fields where deep learning is active

Image recognition

Natural language processing

Feed-forward neural networks for NLP

Deep learning for NLP

Recurrent neural networks

Long short term memory networks

The difficulties of deep learning

The approaches to maximizing deep learning possibilities and abilities

Field-oriented approach

Medicine

Automobiles

Advert technologies

Profession or practice

Sports

Breakdown-oriented approach

Output-oriented approach

Summary

7. Other Important Deep Learning Libraries

Theano

TensorFlow

Caffe

Summary

8. What's Next?

Breaking news about deep learning

Expected next actions

Useful news sources for deep learning

Summary

2. Machine Learning in Java

1. Applied Machine Learning Quick Start

Machine learning and data science

What kind of problems can machine learning solve?

Applied machine learning workflow

Data and problem definition

Measurement scales

Data collection

Find or observe data

Generate data

Sampling traps

Data pre-processing

Data cleaning

Fill missing values

Remove outliers

Data transformation

Data reduction

Unsupervised learning

Find similar items

Euclidean distances

Non-Euclidean distances

The curse of dimensionality

Clustering

Supervised learning

Classification

Decision tree learning

Probabilistic classifiers

Kernel methods

Artificial neural networks

Ensemble learning

Evaluating classification

Precision and recall

Roc curves

Regression

Linear regression

Evaluating regression

Mean squared error

Mean absolute error

Correlation coefficient

Generalization and evaluation

Underfitting and overfitting

Train and test sets

Cross-validation

Leave-one-out validation

Stratification

Summary

2. Java Libraries and Platforms for Machine Learning

The need for Java

Machine learning libraries

Weka

Java machine learning

Apache Mahout

Apache Spark

Deeplearning4j

MALLET

Comparing libraries

Building a machine learning application

Traditional machine learning architecture

Dealing with big data

Big data application architecture

Summary

3. Basic Algorithms – Classification, Regression, and Clustering

Before you start

Classification

Data

Loading data

Feature selection

Learning algorithms

Classify new data

Evaluation and prediction error metrics

Confusion matrix

Choosing a classification algorithm

Regression

Loading the data

Analyzing attributes

Building and evaluating regression model

Linear regression

Regression trees

Tips to avoid common regression problems

Clustering

Clustering algorithms

Evaluation

Summary

4. Customer Relationship Prediction with Ensembles

Customer relationship database

Challenge

Dataset

Evaluation

Basic naive Bayes classifier baseline

Getting the data

Loading the data

Basic modeling

Evaluating models

Implementing naive Bayes baseline

Advanced modeling with ensembles

Before we start

Data pre-processing

Attribute selection

Model selection

Performance evaluation

Summary

5. Affinity Analysis

Market basket analysis

Affinity analysis

Association rule learning

Basic concepts

Database of transactions

Itemset and rule

Support

Confidence

Apriori algorithm

FP-growth algorithm

The supermarket dataset

Discover patterns

Apriori

FP-growth

Other applications in various areas

Medical diagnosis

Protein sequences

Census data

Customer relationship management

IT Operations Analytics

Summary

6. Recommendation Engine with Apache Mahout

Basic concepts

Key concepts

User-based and item-based analysis

Approaches to calculate similarity

Collaborative filtering

Content-based filtering

Hybrid approach

Exploitation versus exploration

Getting Apache Mahout

Configuring Mahout in Eclipse with the Maven plugin

Building a recommendation engine

Book ratings dataset

Loading the data

Loading data from file

Loading data from database

In-memory database

Collaborative filtering

User-based filtering

Item-based filtering

Adding custom rules to recommendations

Evaluation

Online learning engine

Content-based filtering

Summary

7. Fraud and Anomaly Detection

Suspicious and anomalous behavior detection

Unknown-unknowns

Suspicious pattern detection

Anomalous pattern detection

Analysis types

Pattern analysis

Transaction analysis

Plan recognition

Fraud detection of insurance claims

Dataset

Modeling suspicious patterns

Vanilla approach

Dataset rebalancing

Anomaly detection in website traffic

Dataset

Anomaly detection in time series data

Histogram-based anomaly detection

Loading the data

Creating histograms

Density based k-nearest neighbors

Summary

8. Image Recognition with Deeplearning4j

Introducing image recognition

Neural networks

Perceptron

Feedforward neural networks

Autoencoder

Restricted Boltzmann machine

Deep convolutional networks

Image classification

Deeplearning4j

Getting DL4J

MNIST dataset

Loading the data

Building models

Building a single-layer regression model

Building a deep belief network

Build a Multilayer Convolutional Network

Summary

9. Activity Recognition with Mobile Phone Sensors

Introducing activity recognition

Mobile phone sensors

Activity recognition pipeline

The plan

Collecting data from a mobile phone

Installing Android Studio

Loading the data collector

Feature extraction

Collecting training data

Building a classifier

Reducing spurious transitions

Plugging the classifier into a mobile app

Summary

10. Text Mining with Mallet – Topic Modeling and Spam Detection

Introducing text mining

Topic modeling

Text classification

Installing Mallet

Working with text data

Importing data

Importing from directory

Importing from file

Pre-processing text data

Topic modeling for BBC news

BBC dataset

Modeling

Evaluating a model

Reusing a model

Saving a model

Restoring a model

E-mail spam detection

E-mail spam dataset

Feature generation

Training and testing

Model performance

Summary

11. What is Next?

Machine learning in real life

Noisy data

Class unbalance

Feature selection is hard

Model chaining

Importance of evaluation

Getting models into production

Model maintenance

Standards and markup languages

CRISP-DM

SEMMA methodology

Predictive Model Markup Language

Machine learning in the cloud

Machine learning as a service

Web resources and competitions

Datasets

Online courses

Competitions

Websites and blogs

Venues and conferences

Summary

A. References

3. Neural Network Programming with Java, Second Edition

1. Getting Started with Neural Networks

Discovering neural networks

Why artificial neural networks?

How neural networks are arranged

The very basic element – artificial neuron

Giving life to neurons – activation function

The flexible values – weights

An extra parameter – bias

The parts forming the whole – layers

Learning about neural network architectures

Monolayer networks

Multilayer networks

Feedforward networks

Feedback networks

From ignorance to knowledge – learning process

Let the coding begin! Neural networks in practice

The neuron class

The NeuralLayer class

The ActivationFunction interface

The neural network class

Time to play!

Summary

2. Getting Neural Networks to Learn

Learning ability in neural networks

How learning helps solving problems

Learning paradigms

Supervised learning

Unsupervised learning

The learning process

The cost function finding the way down to the optimum

Learning in progress - weight update

Calculating the cost function

General error and overall error

Can the neural network learn forever? When is it good to stop?

Examples of learning algorithms

The delta rule

The learning rate

Implementing the delta rule

The core of the delta rule learning - train and calcNewWeight methods

Another learning algorithm - Hebbian learning

Adaline

Time to see the learning in practice!

Teaching the neural network – the training dataset

Amazing, it learned! Or, did it really? A further step – testing

Overfitting and overtraining

Summary

3. Perceptrons and Supervised Learning

Supervised learning – teaching the neural net

Classification – finding the appropriate class

Regression – mapping real inputs to outputs

A basic neural architecture – perceptrons

Applications and limitations

Linear separation

The XOR case

Multi-layer perceptrons

MLP properties

MLP weights

Recurrent MLP

Coding an MLP

Learning in MLPs

Backpropagation algorithm

The momentum

Coding the backpropagation

Levenberg-Marquardt algorithm

Coding the Levenberg-Marquardt with matrix algebra

Extreme learning machines

Practical example 1 – the XOR case with delta rule and backpropagation

Practical example 2 – predicting enrolment status

Summary

4. Self-Organizing Maps

Neural networks unsupervised learning

Unsupervised learning algorithms

Competitive learning

Competitive layer

Kohonen self-organizing maps

Extending the neural network code to Kohonen

Zero-dimensional SOM

One-dimensional SOM

Two-dimensional SOM

2D competitive layer

SOM learning algorithm

Effect of neighboring neurons – the neighborhood function

The learning rate

A new class for competitive learning

Visualizing the SOMs

Plotting 2D training datasets and neuron weights

Testing Kohonen learning

Summary

5. Forecasting Weather

Neural networks for regression problems

Loading/selecting data

Building auxiliary classes

Getting a dataset from a CSV file

Building time series

Dropping NaNs

Getting weather data

Weather variables

Choosing input and output variables

Preprocessing

Normalization

Adapting NeuralDataSet to handle normalization

Adapting the learning algorithm to normalization

Java implementation of weather forecasting

Collecting weather data

Delaying variables

Loading the data and beginning to play!

Let's perform a correlation analysis

Creating neural networks

Training and test

Training the neural network

Plotting the error

Viewing the neural network output

Empirical design of neural networks

Designing experiments

Results and simulations

Summary

6. Classifying Disease Diagnosis

Foundations of classification problems

Categorical data

Working with categorical data

Logistic regression

Multiple classes versus binary classes

Confusion matrix

Sensitivity and specificity

Implementing a confusion matrix

Neural networks for classification

Disease diagnosis with neural networks

Breast cancer

Diabetes

Summary

7. Clustering Customer Profiles

Clustering tasks

Cluster analysis

Cluster evaluation and validation

Implementation

External validation

Applied unsupervised learning

Kohonen neural network

Profiling

Pre-processing

Implementation in Java

Card – credit analysis for customer profiling

Product profiling

How many clusters?

Summary

8. Text Recognition

Pattern recognition

Defined classes

Undefined classes

Neural networks in pattern recognition

Data pre-processing

Text recognition (optical character recognition)

Digit recognition

Digit representation

Implementation in Java

Generating data

Neural architecture

Experiments

Results

Summary

9. Optimizing and Adapting Neural Networks

Common issues in neural network implementations

Input selection

Data correlation

Transforming data

Dimensionality reduction

Data filtering

Cross-validation

Structure selection

Online retraining

Stochastic online learning

Implementation

Application

Adaptive neural networks

Adaptive resonance theory

Implementation

Summary

10. Current Trends in Neural Networks

Deep learning

Deep architectures

How to implement deep learning in Java

Hybrid systems

Neuro-fuzzy

Neuro-genetic

Implementing a hybrid neural network

Summary

A. References

Chapter 1: Getting Started with Neural Networks

Chapter 2: Getting Neural Networks to Learn

Chapter 3: Perceptrons and Supervised Learning

Chapter 4: Self-Organizing Maps

Chapter 5: Forecasting Weather

Chapter 6: Classifying Disease Diagnosis

Chapter 7: Clustering Customer Profiles

Chapter 8: Text Recognition

Chapter 9: Optimizing and Adapting Neural Networks

Chapter 10: Current Trends in Neural Networks

Bibliography

Index