Expert Python Programming - Second Edition电子书

Expert Python Programming Second Edition

Expert Python Programming Second Edition

Credits

About the Authors

About the Reviewer

www.PacktPub.com

eBooks, discount offers, and more

Why subscribe?

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Current Status of Python

Where are we now and where we are going?

Why and how does Python change?

Getting up to date with changes – PEP documents

Python 3 adoption at the time of writing this book

The main differences between Python 3 and Python 2

Why should I care?

The main syntax differences and common pitfalls

Syntax changes

Changes in the standard library

Changes in datatypes and collections

The popular tools and techniques used for maintaining cross-version compatibility

Not only CPython

Why should I care?

Stackless Python

Jython

IronPython

PyPy

Modern approaches to Python development

Application-level isolation of Python environments

Why isolation?

Popular solutions

virtualenv

venv

buildout

Which one to choose?

System-level environment isolation

Virtual development environments using Vagrant

Containerization versus virtualization

Popular productivity tools

Custom Python shells – IPython, bpython, ptpython, and so on

Setting up the PYTHONSTARTUP environment variable

IPython

bpython

ptpython

Interactive debuggers

Useful resources

Summary

2. Syntax Best Practices – below the Class Level

Python's built-in types

Strings and bytes

Implementation details

String concatenation

Collections

Lists and tuples

Implementation details

List comprehensions

Other idioms

Dictionaries

Implementation details

Weaknesses and alternatives

Sets

Implementation details

Beyond basic collections – the collections module

Advanced syntax

Iterators

The yield statement

Decorators

General syntax and possible implementations

As a function

As a class

Parametrizing decorators

Introspection preserving decorators

Usage and useful examples

Argument checking

Caching

Proxy

Context provider

Context managers – the with statement

General syntax and possible implementations

As a class

As a function – the contextlib module

Other syntax elements you may not know yet

The for … else … statement

Function annotations

The general syntax

The possible uses

Summary

3. Syntax Best Practices – above the Class Level

Subclassing built-in types

Accessing methods from superclasses

Old-style classes and super in Python 2

Understanding Python's Method Resolution Order

super pitfalls

Mixing super and explicit class calls

Heterogeneous arguments

Best practices

Advanced attribute access patterns

Descriptors

Real-life example – lazily evaluated attributes

Properties

Slots

Metaprogramming

Decorators – a method of metaprogramming

Class decorators

Using the __new__() method to override instance creation process

Metaclasses

The general syntax

New Python 3 syntax for metaclasses

Metaclass usage

Metaclass pitfalls

Some tips on code generation

exec, eval, and compile

Abstract Syntax Tree

Import hooks

Projects using code generation patterns

Falcon's compiled router

Hy

Summary

4. Choosing Good Names

PEP 8 and naming best practices

Why and when to follow PEP 8?

Beyond PEP 8 – team-specific style guidelines

Naming styles

Variables

Constants

Naming and usage

Public and private variables

Functions and methods

The private controversy

Special methods

Arguments

Properties

Classes

Modules and packages

The naming guide

Using the has or is prefix for Boolean elements

Using plurals for variables that are collections

Using explicit names for dictionaries

Avoiding generic names

Avoiding existing names

Best practices for arguments

Building arguments by iterative design

Trust the arguments and your tests

Using *args and **kwargs magic arguments carefully

Class names

Module and package names

Useful tools

Pylint

pep8 and flake8

Summary

5. Writing a Package

Creating a package

The confusing state of Python packaging tools

The current landscape of Python packaging thanks to PyPA

Tool recommendations

Project configuration

setup.py

setup.cfg

MANIFEST.in

Most important metadata

Trove classifiers

Common patterns

Automated inclusion of version string from package

README file

Managing dependencies

The custom setup command

Working with packages during development

setup.py install

Uninstalling packages

setup.py develop or pip -e

Namespace packages

Why is it useful?

PEP 420 – implicit namespace packages

Namespace packages in previous Python versions

Uploading a package

PyPI – Python Package Index

Uploading to PyPI – or other package index

.pypirc

Source packages versus built packages

sdist

bdist and wheels

Standalone executables

When are standalone executables useful?

Popular tools

PyInstaller

cx_Freeze

py2exe and py2app

Security of Python code in executable packages

Making decompilation harder

Summary

6. Deploying Code

The Twelve-Factor App

Deployment automation using Fabric

Your own package index or index mirror

PyPI mirroring

Deployment using a package

Common conventions and practices

The filesystem hierarchy

Isolation

Using process supervision tools

Application code should be run in user space

Using reverse HTTP proxies

Reloading processes gracefully

Code instrumentation and monitoring

Logging errors – sentry/raven

Monitoring system and application metrics

Dealing with application logs

Basic low-level log practices

Tools for log processing

Summary

7. Python Extensions in Other Languages

Different language means – C or C++

How do extensions in C or C++ work

Why you might want to use extensions

Improving performance in critical code sections

Integrating existing code written in different languages

Integrating third-party dynamic libraries

Creating custom datatypes

Writing extensions

Pure C extensions

A closer look at Python/C API

Calling and binding conventions

Exception handling

Releasing GIL

Reference counting

Cython

Cython as a source to source compiler

Cython as a language

Challenges

Additional complexity

Debugging

Interfacing with dynamic libraries without extensions

ctypes

Loading libraries

Calling C functions using ctypes

Passing Python functions as C callbacks

CFFI

Summary

8. Managing Code

Version control systems

Centralized systems

Distributed systems

Distributed strategies

Centralized or distributed?

Use Git if you can

Git flow and GitHub flow

Continuous development processes

Continuous integration

Testing every commit

Merge testing through CI

Matrix testing

Continuous delivery

Continuous deployment

Popular tools for continuous integration

Jenkins

Buildbot

Travis CI

GitLab CI

Choosing the right tool and common pitfalls

Problem 1 – too complex build strategies

Problem 2 – too long building time

Problem 3 – external job definitions

Problem 4 – lack of isolation

Summary

9. Documenting Your Project

The seven rules of technical writing

Write in two steps

Target the readership

Use a simple style

Limit the scope of information

Use realistic code examples

Use a light but sufficient approach

Use templates

A reStructuredText primer

Section structure

Lists

Inline markup

Literal block

Links

Building the documentation

Building the portfolio

Design

Usage

Recipe

Tutorial

Module helper

Operations

Making your own portfolio

Building the landscape

Producer's layout

Consumer's layout

Working on the index pages

Registering module helpers

Adding index markers

Cross-references

Documentation building and continuous integration

Summary

10. Test-Driven Development

I don't test

Test-driven development principles

Preventing software regression

Improving code quality

Providing the best developer documentation

Producing robust code faster

What kind of tests?

Acceptance tests

Unit tests

Functional tests

Integration tests

Load and performance testing

Code quality testing

Python standard test tools

unittest

doctest

I do test

unittest pitfalls

unittest alternatives

nose

Test runner

Writing tests

Writing test fixtures

Integration with setuptools and a plug-in system

Wrap-up

py.test

Writing test fixtures

Disabling test functions and classes

Automated distributed tests

Wrap-up

Testing coverage

Fakes and mocks

Building a fake

Using mocks

Testing environment and dependency compatibility

Dependency matrix testing

Document-driven development

Writing a story

Summary

11. Optimization – General Principles and Profiling Techniques

The three rules of optimization

Make it work first

Work from the user's point of view

Keep the code readable and maintainable

Optimization strategy

Find another culprit

Scale the hardware

Writing a speed test

Finding bottlenecks

Profiling CPU usage

Macro-profiling

Micro-profiling

Measuring Pystones

Profiling memory usage

How Python deals with memory

Profiling memory

objgraph

C code memory leaks

Profiling network usage

Summary

12. Optimization – Some Powerful Techniques

Reducing the complexity

Cyclomatic complexity

The big O notation

Simplifying

Searching in a list

Using a set instead of a list

Cut the external calls, reduce the workload

Using collections

deque

defaultdict

namedtuple

Using architectural trade-offs

Using heuristics and approximation algorithms

Using task queues and delayed processing

Using probabilistic data structures

Caching

Deterministic caching

Nondeterministic caching

Cache services

Memcached

Summary

13. Concurrency

Why concurrency?

Multithreading

What is multithreading?

How Python deals with threads

When should threading be used?

Building responsive interfaces

Delegating work

Multiuser applications

An example of a threaded application

Using one thread per item

Using a thread pool

Using two-way queues

Dealing with errors and rate limiting

Multiprocessing

The built-in multiprocessing module

Using process pools

Using multiprocessing.dummy as a multithreading interface

Asynchronous programming

Cooperative multitasking and asynchronous I/O

Python async and await keywords

asyncio in older versions of Python

A practical example of asynchronous programming

Integrating nonasynchronous code with async using futures

Executors and futures

Using executors in an event loop

Summary

14. Useful Design Patterns

Creational patterns

Singleton

Structural patterns

Adapter

Interfaces

Using zope.interface

Using function annotations and abstract base classes

Using collections.abc

Proxy

Facade

Behavioral patterns

Observer

Visitor

Template

Summary

Index