Expert Python Programming电子书

Dedication

About Packt

Why subscribe?

Packt.com

Contributors

About the authors

About the reviewer

Packt is searching for authors like you

Preface

Who this book is for

What this book covers

To get the most out of this book

Download the example code files

Download the color images

Conventions used

Get in touch

Reviews

Section 1: Before You Start

Current Status of Python

Technical requirements

Where are we now and where we are going to?

Why and how Python changes

Being up-to-date with changes ­by following PEP documents

Python 3 adoption at the time of writing this book

The main differences between Python 3 and Python 2

Why should I care?

The main syntax differences and common pitfalls

Syntax changes

Changes in the standard library

Changes in data types and collections and string literals

The popular tools and techniques used for maintaining cross-version compatibility

Not only CPython

Why should I care?

Stackless Python

Jython

IronPython

PyPy

MicroPython

Useful resources

Summary

Modern Python Development Environments

Technical requirements

Installing additional Python packages using pip

Isolating the runtime environment

Application-level isolation versus system-level isolation

Python's venv

venv versus virtualenv

System-level environment isolation

Virtual development environments using Vagrant

Virtual environments using Docker

Containerization versus virtualization

Writing your first Dockerfile

Running containers

Setting up complex environments

Useful Docker recipes for Python

Reducing the size of containers

Addressing services inside of a Compose environment

Communicating between multiple Compose environments

Popular productivity tools

Custom Python shells – ipython, bpython, ptpython, and so on

Setting up the PYTHONSTARTUP environment variable

IPython

bpython

ptpython

Incorporating shells in your own scripts and programs

Interactive debuggers

Summary

Section 2: Python Craftsmanship

Modern Syntax Elements - Below the Class Level

Technical requirements

Python's built-in types

Strings and bytes

Implementation details

String concatenation

Constant folding, the peephole optimizer, and the AST optimizer

String formatting with f-strings

Containers

Lists and tuples

Implementation details

List comprehensions

Other idioms

Dictionaries

Implementation details

Weaknesses and alternatives

Sets

Implementation details

Supplemental data types and containers

Specialized data containers from the collections module

Symbolic enumeration with the enum module

Advanced syntax

Iterators

Generators and yield statements

Decorators

General syntax and possible implementations

As a function

As a class

Parametrizing decorators

Introspection preserving decorators

Usage and useful examples

Argument checking

Caching

Proxy

Context provider

Context managers – the with statement

The general syntax and possible implementations

As a class

As a function – the contextlib module

Functional-style features of Python

What is functional programming?

Lambda functions

map(), filter(), and reduce()

Partial objects and partial() functions

Generator expressions

Function and variable annotations

The general syntax

The possible uses

Static type checking with mypy

Other syntax elements you may not know of yet

The for ... else ... statement

Keyword-only arguments

Summary

Modern Syntax Elements - Above the Class Level

Technical requirements

The protocols of the Python language – dunder methods and attributes

Reducing boilerplate with data classes

Subclassing built-in types

MRO and accessing methods from superclasses

Old-style classes and super in Python 2

Understanding Python's Method Resolution Order

Super pitfalls

Mixing super and explicit class calls

Heterogeneous arguments

Best practices

Advanced attribute access patterns

Descriptors

Real-life example – lazily evaluated attributes

Properties

Slots

Summary

Elements of Metaprogramming

Technical requirements

What is metaprogramming?

Decorators – a method of metaprogramming

Class decorators

Using __new__() for overriding the instance creation process

Metaclasses

The general syntax

New Python 3 syntax for metaclasses

Metaclass usage

Metaclass pitfalls

Code generation

exec, eval, and compile

Abstract syntax tree (AST)

Import hooks

Projects that use code generation patterns

Falcon's compiled router

Hy

Summary

Choosing Good Names

Technical requirements

PEP 8 and naming best practices

Why and when to follow PEP 8?

Beyond PEP 8 – Team-specific style guidelines

Naming styles

Variables

Constants

Naming and usage

Public and private variables

Functions and methods

The private controversy

Special methods

Arguments

Properties

Classes

Modules and packages

The naming guide

Using the has/is prefixes for Boolean elements

Using plurals for variables that are collections

Using explicit names for dictionaries

Avoid generic names and redundancy

Avoiding existing names

Best practices for arguments

Building arguments by iterative design

Trusting the arguments and your tests

Using *args and **kwargs magic arguments carefully

Class names

Module and package names

Useful tools

Pylint

pycodestyle and flake8

Summary

Writing a Package

Technical requirements

Creating a package

The confusing state of Python packaging tools

The current landscape of Python packaging thanks to PyPA

Tool recommendations

Project configuration

setup.py

setup.cfg

MANIFEST.in

Most important metadata

Trove classifiers

Common patterns

Automated inclusion of version string from package

README file

Managing dependencies

The custom setup command

Working with packages during development

setup.py install

Uninstalling packages

setup.py develop or pip -e

Namespace packages

Why is it useful?

PEP 420 - implicit namespace packages

Namespace packages in previous Python versions

Uploading a package

PyPI - Python Package Index

Uploading to PyPI - or other package index

.pypirc

Source packages versus built packages

sdist

bdist and wheels

Standalone executables

When standalone executables useful?

Popular tools

PyInstaller

cx_Freeze

py2exe and py2app

Security of Python code in executable packages

Making decompilation harder

Summary

Deploying the Code

Technical requirements

The Twelve-Factor App

Various approaches to deployment automation

Using Fabric for deployment automation

Your own package index or index mirror

PyPI mirroring

Bundling additional resources with your Python package

Common conventions and practices

The filesystem hierarchy

Isolation

Using process supervision tools

Application code running in user space

Using reverse HTTP proxies

Reloading processes gracefully

Code instrumentation and monitoring

Logging errors – Sentry/Raven

Monitoring system and application metrics

Dealing with application logs

Basic low-level log practices

Tools for log processing

Summary

Python Extensions in Other Languages

Technical requirements

Differentiating between the C and C++ languages

Loading extensions in C or C++

The need to use extensions

Improving the performance in critical code sections

Integrating existing code written in different languages

Integrating third-party dynamic libraries

Creating custom datatypes

Writing extensions

Pure C extensions

A closer look at Python/C API

Calling and binding conventions

Exception handling

Releasing GIL

Reference counting

Writing extensions with Cython

Cython as a source-to-source compiler

Cython as a language

Challenges with using extensions

Additional complexity

Debugging

Interfacing with dynamic libraries without extensions

The ctypes module

Loading libraries

Calling C functions using ctypes

Passing Python functions as C callbacks

CFFI

Summary

Section 3: Quality over Quantity

Managing Code

Technical requirements

Working with a version control system

Centralized systems

Distributed systems

Distributed strategies

Centralized or distributed?

Use Git if you can

GitFlow and GitHub Flow

Setting up continuous development processes

Continuous integration

Testing every commit

Merge testing through CI

Matrix testing

Continuous delivery

Continuous deployment

Popular tools for continuous integration

Jenkins

Buildbot

Travis CI

GitLab CI

Choosing the right tool and common pitfalls

Problem 1 – Complex build strategies

Problem 2 – Long building time

Problem 3 – External job definitions

Problem 4 – Lack of isolation

Summary

Documenting Your Project

Technical requirements

The seven rules of technical writing

Write in two steps

Target the readership

Use a simple style

Limit the scope of information

Use realistic code examples

Use a light but sufficient approach

Use templates

Documentation as code

Using Python docstrings

Popular markup languages and styles for documentation

Popular documentation generators for Python libraries

Sphinx

Working with the index pages

Registering module helpers

Adding index markers

Cross-references

MkDocs

Documentation building and continuous integration

Documenting web APIs

Documentation as API prototype with API Blueprint

Self-documenting APIs with Swagger/OpenAPI

Building a well-organized documentation system

Building documentation portfolio

Design

Usage

Recipe

Tutorial

Module helper

Operations

Your very own documentation portfolio

Building a documentation landscape

Producer's layout

Consumer's layout

Summary

Test-Driven Development

Technical requirements

I don't test

Three simple steps of test-driven development

Preventing software regression

Improving code quality

Providing the best developer documentation

Producing robust code faster

What kind of tests?

Unit tests

Acceptance tests

Functional tests

Integration tests

Load and performance testing

Code quality testing

Python standard test tools

unittest

doctest

I do test

unittest pitfalls

unittest alternatives

nose

Test runner

Writing tests

Writing test fixtures

Integration with setuptools and plugin system

Wrap-up

py.test

Writing test fixtures

Disabling test functions and classes

Automated distributed tests

Wrap-up

Testing coverage

Fakes and mocks

Building a fake

Using mocks

Testing environment and dependency compatibility

Dependency matrix testing

Document-driven development

Writing a story

Summary

Section 4: Need for Speed

Optimization - Principles and Profiling Techniques

Technical requirements

The three rules of optimization

Making it work first

Working from the user's point of view

Keeping the code readable and maintainable

Optimization strategy

Looking for another culprit

Scaling the hardware

Writing a speed test

Finding bottlenecks

Profiling CPU usage

Macro-profiling

Micro-profiling

Profiling memory usage

How Python deals with memory

Profiling memory

objgraph

C code memory leaks

Profiling network usage

Tracing network transactions

Summary

Optimization - Some Powerful Techniques

Technical requirements

Defining complexity

Cyclomatic complexity

The big O notation

Reducing complexity by choosing proper data structures

Searching in a list

Using sets

Using collections

deque

defaultdict

namedtuple

Using architectural trade-offs

Using heuristics and approximation algorithms

Using task queues and delayed processing

Using probabilistic data structures

Caching

Deterministic caching

Non-deterministic caching

Cache services

Memcached

Summary

Concurrency

Technical requirements

Why concurrency?

Multithreading

What is multithreading?

How Python deals with threads

When should we use threading?

Building responsive interfaces

Delegating work

Multiuser applications

An example of a threaded application

Using one thread per item

Using a thread pool

Using two-way queues

Dealing with errors and rate limiting

Multiprocessing

The built-in multiprocessing module

Using process pools

Using multiprocessing.dummy as the multithreading interface

Asynchronous programming

Cooperative multitasking and asynchronous I/O

Python async and await keywords

asyncio in older versions of Python

A practical example of asynchronous programming

Integrating non-asynchronous code with async using futures

Executors and futures

Using executors in an event loop

Summary

Section 5: Technical Architecture

Event-Driven and Signal Programming

Technical requirements

What exactly is event-driven programming?

Event-driven != asynchronous

Event-driven programming in GUIs

Event-driven communication

Various styles of event-driven programming

Callback-based style

Subject-based style

Topic-based style

Event-driven architectures

Event and message queues

Summary

Useful Design Patterns

Technical requirements

Creational patterns

Singleton

Structural patterns

Adapter

Interfaces

Using zope.interface

Using function annotations and abstract base classes

Using collections.abc

Proxy

Facade

Behavioral patterns

Observer

Visitor

Template

Summary

reStructuredText Primer

reStructuredText

Section structure

Lists

Inline markup

Literal block

Links

Other Books You May Enjoy

Leave a review - let other readers know what you think