Mastering Python High Performance电子书

Mastering Python High Performance

Table of Contents

Mastering Python High Performance

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

1. Profiling 101

What is profiling?

Event-based profiling

Statistical profiling

The importance of profiling

What can we profile?

Execution time

Where are the bottlenecks?

Memory consumption and memory leaks

The risk of premature optimization

Running time complexity

Constant time – O(1)

Linear time – O(n)

Logarithmic time – O(log n)

Linearithmic time – O(nlog n)

Factorial time – O(n!)

Quadratic time – O(n^)

Profiling best practices

Build a regression-test suite

Mind your code

Be patient

Gather as much data as you can

Preprocess your data

Visualize your data

Summary

2. The Profilers

Getting to know our new best friends: the profilers

cProfile

A note about limitations

The API provided

The Stats class

Profiling examples

Fibonacci again

Tweet stats

line_profiler

kernprof

Some things to consider about kernprof

Profiling examples

Back to Fibonacci

Inverted index

getOffsetUpToWord

getWords

list2dict

readFileContent

saveIndex

__start__

getOffsetUpToWord

getWords

list2dict

saveIndex

Summary

3. Going Visual – GUIs to Help Understand Profiler Output

KCacheGrind – pyprof2calltree

Installation

Usage

A profiling example – TweetStats

A profiling example – Inverted Index

RunSnakeRun

Installation

Usage

Profiling examples – the lowest common multiplier

A profiling example – search using the inverted index

Summary

4. Optimize Everything

Memoization / lookup tables

Performing a lookup on a list or linked list

Simple lookup on a dictionary

Binary search

Use cases for lookup tables

Usage of default arguments

List comprehension and generators

ctypes

Loading your own custom C library

Loading a system library

String concatenation

Other tips and tricks

Summary

5. Multithreading versus Multiprocessing

Parallelism versus concurrency

Multithreading

Threads

Creating a thread with the thread module

Working with the threading module

Interthread communication with events

Multiprocessing

Multiprocessing with Python

Exit status

Process pooling

Interprocess communication

Pipes

Events

Summary

6. Generic Optimization Options

PyPy

Installing PyPy

A Just-in-time compiler

Sandboxing

Optimizing for the JIT

Think of functions

Consider using cStringIO to concatenate strings

Actions that disable the JIT

Code sample

Cython

Installing Cython

Building a Cython module

Calling C functions

Solving naming conflicts

Defining types

Defining types during function definitions

A Cython example

When to define a type

Limitations

Generator expressions

Comparison of char* literals

Tuples as function arguments

Stack frames

How to choose the right option

When to go with Cython

When to go with PyPy

Summary

7. Lightning Fast Number Crunching with Numba, Parakeet, and pandas

Numba

Installation

Using Numba

Numba's code generation

Eager compilation

Other configuration settings

No GIL

NoPython mode

Running your code on the GPU

The pandas tool

Installing pandas

Using pandas for data analysis

Parakeet

Installing Parakeet

How does Parakeet work?

Summary

8. Putting It All into Practice

The problem to solve

Getting data from the Web

Postprocessing the data

The initial code base

Analyzing the code

Scraper

Analyzer

Summary

Index