Parallel Programming with Python电子书

Parallel Programming with Python

Table of Contents

Parallel Programming with Python

Credits

About the Author

Acknowledgments

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Contextualizing Parallel, Concurrent, and Distributed Programming

Why use parallel programming?

Exploring common forms of parallelization

Communicating in parallel programming

Understanding shared state

Understanding message passing

Identifying parallel programming problems

Deadlock

Starvation

Race conditions

Discovering Python's parallel programming tools

The Python threading module

The Python multiprocessing module

The parallel Python module

Celery – a distributed task queue

Taking care of Python GIL

Summary

2. Designing Parallel Algorithms

The divide and conquer technique

Using data decomposition

Decomposing tasks with pipeline

Processing and mapping

Identifying independent tasks

Identifying the tasks that require data exchange

Load balance

Summary

3. Identifying a Parallelizable Problem

Obtaining the highest Fibonacci value for multiple inputs

Crawling the Web

Summary

4. Using the threading and concurrent.futures Modules

Defining threads

Advantages and disadvantages of using threads

Understanding different kinds of threads

Defining the states of a thread

Choosing between threading and _thread

Using threading to obtain the Fibonacci series term with multiple inputs

Crawling the Web using the concurrent.futures module

Summary

5. Using Multiprocessing and ProcessPoolExecutor

Understanding the concept of a process

Understanding the process model

Defining the states of a process

Implementing multiprocessing communication

Using multiprocessing.Pipe

Understanding multiprocessing.Queue

Using multiprocessing to compute Fibonacci series terms with multiple inputs

Crawling the Web using ProcessPoolExecutor

Summary

6. Utilizing Parallel Python

Understanding interprocess communication

Exploring named pipes

Using named pipes with Python

Writing in a named pipe

Reading named pipes

Discovering PP

Using PP to calculate the Fibonacci series term on SMP architecture

Using PP to make a distributed Web crawler

Summary

7. Distributing Tasks with Celery

Understanding Celery

Why use Celery?

Understanding Celery's architecture

Working with tasks

Discovering message transport (broker)

Understanding workers

Understanding result backends

Setting up the environment

Setting up the client machine

Setting up the server machine

Dispatching a simple task

Using Celery to obtain a Fibonacci series term

Defining queues by task types

Using Celery to make a distributed Web crawler

Summary

8. Doing Things Asynchronously

Understanding blocking, nonblocking, and asynchronous operations

Understanding blocking operations

Understanding nonblocking operations

Understanding asynchronous operations

Understanding event loop

Polling functions

Using event loops

Using asyncio

Understanding coroutines and futures

Using coroutine and asyncio.Future

Using asyncio.Task

Using an incompatible library with asyncio

Summary

Index