Ipython Interactive Computing and Visualization Cookbook电子书

IPython Interactive Computing and Visualization Cookbook

Table of Contents

IPython Interactive Computing and Visualization Cookbook

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why Subscribe?

Free Access for Packt account holders

Preface

What this book is

What this book covers

Part 1 – Advanced High-Performance Interactive Computing

Part 2 – Standard Methods in Data Science and Applied Mathematics

What you need for this book

Installing Python

GitHub repositories

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images

Errata

Piracy

Questions

1. A Tour of Interactive Computing with IPython

Introduction

What is IPython?

A brief historical retrospective on Python as a scientific environment

What's new in IPython 2.0?

Roadmap for IPython 3.0 and 4.0

References

Introducing the IPython notebook

Getting ready

How to do it...

There's more...

See also

Getting started with exploratory data analysis in IPython

How to do it...

There's more...

See also

Introducing the multidimensional array in NumPy for fast array computations

How to do it...

How it works...

There's more...

See also

Creating an IPython extension with custom magic commands

How to do it...

How it works...

The InteractiveShell class

Loading an extension

There's more...

See also

Mastering IPython's configuration system

How to do it...

How it works...

Configurables

Magics

There's more...

See also

Creating a simple kernel for IPython

Getting ready

How to do it...

How it works...

There's more...

2. Best Practices in Interactive Computing

Introduction

Choosing (or not) between Python 2 and Python 3

How to do it...

Main differences in Python 3 compared to Python 2

Python 2 or Python 3?

Supporting both Python 2 and Python 3

Using 2to3

Writing code that works in Python 2 and Python 3

There's more...

See also

Efficient interactive computing workflows with IPython

How to do it...

The IPython terminal

IPython and text editor

The IPython notebook

Integrated Development Environments

There's more...

See also

Learning the basics of the distributed version control system Git

Getting ready

How to do it…

Creating a local repository

Cloning a remote repository

How it works…

There's more…

See also

A typical workflow with Git branching

Getting ready

How to do it…

Stashing

How it works…

There's more…

See also

Ten tips for conducting reproducible interactive computing experiments

How to do it…

How it works…

There's more...

See also

Writing high-quality Python code

How to do it...

How it works...

There's more...

See also

Writing unit tests with nose

Getting ready

How to do it...

How it works...

There's more...

Test coverage

Workflows with unit testing

Unit testing and continuous integration

Debugging your code with IPython

How to do it...

The post-mortem mode

Step-by-step debugging

There's more...

GUI debuggers

3. Mastering the Notebook

Introduction

What is the notebook?

The notebook ecosystem

Architecture of the IPython notebook

Connecting multiple clients to one kernel

Security in notebooks

References

Teaching programming in the notebook with IPython blocks

Getting ready

How to do it...

There's more...

Converting an IPython notebook to other formats with nbconvert

Getting ready

How to do it...

How it works...

There's more...

Adding custom controls in the notebook toolbar

How to do it...

There's more...

See also

Customizing the CSS style in the notebook

Getting ready

How to do it...

There's more...

See also

Using interactive widgets – a piano in the notebook

Getting ready

How to do it...

How it works...

There's more...

See also

Creating a custom JavaScript widget in the notebook – a spreadsheet editor for pandas

Getting ready

How to do it...

How it works...

There's more...

See also

Processing webcam images in real time from the notebook

Getting ready

How to do it...

How it works...

There's more...

See also

4. Profiling and Optimization

Introduction

Evaluating the time taken by a statement in IPython

How to do it...

How it works...

There's more...

See also

Profiling your code easily with cProfile and IPython

How to do it...

How it works...

"Premature optimization is the root of all evil"

There's more...

See also

Profiling your code line-by-line with line_profiler

Getting ready

How do to it...

How it works...

There's more...

Tracing the step-by-step execution of a Python program

See also

Profiling the memory usage of your code with memory_profiler

Getting ready

How to do it...

How it works...

There's more...

Using memory_profiler for standalone Python programs

Using the %memit magic command in IPython

Other tools

See also

Understanding the internals of NumPy to avoid unnecessary array copying

Getting ready

How to do it...

How it works...

Why are NumPy arrays efficient?

What is the difference between in-place and implicit-copy operations?

Why can't some arrays be reshaped without a copy?

What are NumPy broadcasting rules?

There's more...

See also

Using stride tricks with NumPy

Getting ready

How to do it...

How it works...

See also

Implementing an efficient rolling average algorithm with stride tricks

Getting ready

How to do it...

See also

Making efficient array selections in NumPy

Getting ready

How to do it...

How it works...

There's more...

Processing huge NumPy arrays with memory mapping

How to do it...

How it works...

There's more...

See also

Manipulating large arrays with HDF5 and PyTables

Getting ready

How to do it...

How it works...

There's more...

See also

Manipulating large heterogeneous tables with HDF5 and PyTables

Getting ready

How to do it...

How it works...

There's more...

See also

5. High-performance Computing

Introduction

CPython and concurrent programming

Compiler-related installation instructions

Linux

Mac OS X

Windows

Python 32-bit

Python 64-bit

DLL hell

References

Accelerating pure Python code with Numba and just-in-time compilation

Getting ready

How to do it…

How it works…

There's more…

See also

Accelerating array computations with Numexpr

Getting ready

How to do it…

How it works...

See also

Wrapping a C library in Python with ctypes

Getting ready

How to do it…

How it works…

There's more…

Accelerating Python code with Cython

Getting ready

How to do it…

How it works…

There's more…

See also

Optimizing Cython code by writing less Python and more C

How to do it…

How it works…

There's more…

See also

Releasing the GIL to take advantage of multicore processors with Cython and OpenMP

Getting ready

How to do it…

How it works…

See also

Writing massively parallel code for NVIDIA graphics cards (GPUs) with CUDA

Getting ready

How to do it...

How it works…

There's more…

See also

Writing massively parallel code for heterogeneous platforms with OpenCL

Getting ready

How to do it…

How it works…

There's more…

See also

Distributing Python code across multiple cores with IPython

How to do it…

How it works…

There's more…

Dependent parallel tasks

Alternative parallel computing solutions

References

See also

Interacting with asynchronous parallel tasks in IPython

Getting ready

How to do it…

How it works…

There's more…

See also

Parallelizing code with MPI in IPython

Getting ready

How to do it…

How it works…

See also

Trying the Julia language in the notebook

Getting ready

How to do it…

How it works…

There's more…

6. Advanced Visualization

Introduction

Making nicer matplotlib figures with prettyplotlib

Getting ready

How to do it…

How it works…

There's more…

See also

Creating beautiful statistical plots with seaborn

Getting ready

How to do it…

There's more…

See also

Creating interactive web visualizations with Bokeh

Getting ready

How to do it…

There's more…

See also

Visualizing a NetworkX graph in the IPython notebook with D3.js

Getting ready

How to do it…

There's more…

See also

Converting matplotlib figures to D3.js visualizations with mpld3

Getting ready

How to do it…

How it works…

There's more…

See also

Getting started with Vispy for high-performance interactive data visualizations

Getting ready

How to do it…

How it works…

There's more…

Vispy for scientific visualization

7. Statistical Data Analysis

Introduction

What is statistical data analysis?

A bit of vocabulary

Exploration, inference, decision, and prediction

Univariate and multivariate methods

Frequentist and Bayesian methods

Parametric and nonparametric inference methods

Exploring a dataset with pandas and matplotlib

Getting ready

How to do it...

There's more...

Getting started with statistical hypothesis testing – a simple z-test

Getting ready

How to do it...

How it works...

There's more...

See also

Getting started with Bayesian methods

Getting ready

How to do it...

How it works...

Bayes' theorem

Computation of the posterior distribution

Maximum a posteriori estimation

There's more...

Credible interval

Conjugate distributions

Non-informative (objective) prior distributions

See also

Estimating the correlation between two variables with a contingency table and a chi-squared test

Getting ready

How to do it...

How it works...

Pearson's correlation coefficient

Contingency table and chi-squared test

There's more...

See also

Fitting a probability distribution to data with the maximum likelihood method

Getting ready

How to do it...

How it works...

There's more...

See also

Estimating a probability distribution nonparametrically with a kernel density estimation

Getting ready

How to do it...

How it works...

See also

Fitting a Bayesian model by sampling from a posterior distribution with a Markov chain Monte Carlo method

Getting ready

How to do it...

How it works...

There's more...

See also

Analyzing data with the R programming language in the IPython notebook

Getting ready

How to do it...

How it works...

There's more...

See also

8. Machine Learning

Introduction

A bit of vocabulary

Learning from data

Supervised learning

Unsupervised learning

Feature selection and feature extraction

Overfitting, underfitting, and the bias-variance tradeoff

Model selection

Machine learning references

Getting started with scikit-learn

Getting ready

How to do it...

How it works...

The scikit-learn API

Ordinary least squares regression

Polynomial interpolation with linear regression

Ridge regression

Cross-validation and grid search

There's more…

See also

Predicting who will survive on the Titanic with logistic regression

Getting ready

How to do it...

How it works...

There's more...

See also

Learning to recognize handwritten digits with a K-nearest neighbors classifier

How to do it...

How it works...

There's more…

See also

Learning from text – Naive Bayes for Natural Language Processing

Getting ready

How to do it...

How it works...

There's more…

See also

Using support vector machines for classification tasks

How to do it...

How it works...

There's more…

See also

Using a random forest to select important features for regression

How to do it...

How it works...

There's more...

See also

Reducing the dimensionality of a dataset with a principal component analysis

How to do it...

How it works...

There's more…

See also

Detecting hidden structures in a dataset with clustering

How to do it...

How it works...

There's more...

See also

9. Numerical Optimization

Introduction

The objective function

Local and global minima

Constrained and unconstrained optimization

Deterministic and stochastic algorithms

References

Finding the root of a mathematical function

How to do it…

How it works…

There's more…

See also

Minimizing a mathematical function

How to do it…

How it works…

There's more…

See also

Fitting a function to data with nonlinear least squares

How to do it…

How it works…

There's more…

See also

Finding the equilibrium state of a physical system by minimizing its potential energy

How to do it…

How it works…

There's more…

See also

10. Signal Processing

Introduction

Analog and digital signals

The Nyquist–Shannon sampling theorem

Compressed sensing

References

Analyzing the frequency components of a signal with a Fast Fourier Transform

Getting ready

How to do it...

How it works...

The Discrete Fourier Transform

Inverse Fourier Transform

There's more...

See also

Applying a linear filter to a digital signal

Getting ready

How to do it...

How it works...

What are linear filters?

Linear filters and convolutions

The FIR and IIR filters

Filters in the frequency domain

The low-, high-, and band-pass filters

There's more...

See also

Computing the autocorrelation of a time series

Getting ready

How to do it...

How it works...

There's more...

See also

11. Image and Audio Processing

Introduction

Images

Sounds

References

Manipulating the exposure of an image

Getting ready

How to do it...

How it works...

There's more...

See also

Applying filters on an image

How it works...

How it works...

There's more...

See also

Segmenting an image

How to do it...

How it works...

There's more...

See also

Finding points of interest in an image

Getting ready

How to do it...

How it works...

There's more...

Detecting faces in an image with OpenCV

Getting ready

How to do it...

How it works...

There's more...

Applying digital filters to speech sounds

Getting ready

How to do it…

How it works...

There's more...

See also

Creating a sound synthesizer in the notebook

How to do it...

How it works...

There's more...

See also

12. Deterministic Dynamical Systems

Introduction

Types of dynamical systems

Differential equations

References

Plotting the bifurcation diagram of a chaotic dynamical system

How to do it...

There's more...

See also

Simulating an elementary cellular automaton

How to do it...

How it works...

There's more...

Simulating an ordinary differential equation with SciPy

How to do it...

How it works...

There's more...

See also

Simulating a partial differential equation – reaction-diffusion systems and Turing patterns

How to do it...

How it works...

There's more...

See also

13. Stochastic Dynamical Systems

Introduction

References

Simulating a discrete-time Markov chain

How to do it...

How it works...

There's more...

See also

Simulating a Poisson process

How to do it...

How it works...

There's more...

See also

Simulating a Brownian motion

How to do it...

How it works...

There's more...

See also

Simulating a stochastic differential equation

How to do it...

How it works...

There's more...

See also

14. Graphs, Geometry, and Geographic Information Systems

Introduction

Graphs

Problems in graph theory

Random graphs

Graphs in Python

Geometry in Python

Geographical Information Systems in Python

References

Manipulating and visualizing graphs with NetworkX

Getting ready

How to do it…

There's more…

See also

Analyzing a social network with NetworkX

Getting ready

How to do it…

There's more…

See also

Resolving dependencies in a directed acyclic graph with a topological sort

Getting ready

How to do it…

There's more…

Computing connected components in an image

How to do it…

How it works…

There's more…

Computing the Voronoi diagram of a set of points

Getting ready

How to do it…

How it works…

There's more…

See also

Manipulating geospatial data with Shapely and basemap

Getting ready

How to do it…

See also

Creating a route planner for a road network

Getting ready

How to do it…

How it works…

There's more…

15. Symbolic and Numerical Mathematics

Introduction

LaTeX

Diving into symbolic computing with SymPy

Getting ready

How to do it...

How it works...

See also

Solving equations and inequalities

Getting ready

How to do it...

There's more...

Analyzing real-valued functions

Getting ready

How to do it...

There's more...

Computing exact probabilities and manipulating random variables

How to do it...

How it works...

A bit of number theory with SymPy

Getting ready

How to do it...

How it works...

There's more...

Finding a Boolean propositional formula from a truth table

How to do it...

How it works...

There's more...

Analyzing a nonlinear differential system – Lotka-Volterra (predator-prey) equations

Getting ready

How to do it...

How it works...

There's more...

Getting started with Sage

Getting ready

How to do it...

There's more...

See also

Index