Learning DevOps: Continuously Deliver Better Software电子书

Learning DevOps: Continuously Deliver Better Software

Table of Contents

Learning DevOps: Continuously Deliver Better Software

Learning DevOps: Continuously Deliver Better Software

Credits

Preface

What this learning path covers

What you need for this learning path

Who this learning path is for

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

I. Module 1

1. Introduction to DevOps and Continuous Delivery

Introducing DevOps

How fast is fast?

The Agile wheel of wheels

Beware the cargo cult Agile fallacy

DevOps and ITIL

Summary

2. A View from Orbit

The DevOps process and Continuous Delivery – an overview

The developers

The revision control system

The build server

The artifact repository

Package managers

Test environments

Staging/production

Release management

Scrum, Kanban, and the delivery pipeline

Wrapping up – a complete example

Identifying bottlenecks

Summary

3. How DevOps Affects Architecture

Introducing software architecture

The monolithic scenario

Architecture rules of thumb

The separation of concerns

The principle of cohesion

Coupling

Back to the monolithic scenario

A practical example

Three-tier systems

The presentation tier

The logic tier

The data tier

Handling database migrations

Rolling upgrades

Hello world in Liquibase

The changelog file

The pom.xml file

Manual installation

Microservices

Interlude – Conway's Law

How to keep service interfaces forward compatible

Microservices and the data tier

DevOps, architecture, and resilience

Summary

4. Everything is Code

The need for source code control

The history of source code management

Roles and code

Which source code management system?

A word about source code management system migrations

Choosing a branching strategy

Branching problem areas

Artifact version naming

Choosing a client

Setting up a basic Git server

Shared authentication

Hosted Git servers

Large binary files

Trying out different Git server implementations

Docker intermission

Gerrit

Installing the git-review package

The value of history revisionism

The pull request model

GitLab

Summary

5. Building the Code

Why do we build code?

The many faces of build systems

The Jenkins build server

Managing build dependencies

The final artifact

Cheating with FPM

Continuous Integration

Continuous Delivery

Jenkins plugins

The host server

Build slaves

Software on the host

Triggers

Job chaining and build pipelines

A look at the Jenkins filesystem layout

Build servers and infrastructure as code

Building by dependency order

Build phases

Alternative build servers

Collating quality measures

About build status visualization

Taking build errors seriously

Robustness

Summary

6. Testing the Code

Manual testing

Pros and cons with test automation

Unit testing

JUnit in general and JUnit in particular

A JUnit example

Mocking

Test Coverage

Automated integration testing

Docker in automated testing

Arquillian

Performance testing

Automated acceptance testing

Automated GUI testing

Integrating Selenium tests in Jenkins

JavaScript testing

Testing backend integration points

Test-driven development

REPL-driven development

A complete test automation scenario

Manually testing our web application

Running the automated test

Finding a bug

Test walkthrough

Handling tricky dependencies with Docker

Summary

7. Deploying the Code

Why are there so many deployment systems?

Configuring the base OS

Describing clusters

Delivering packages to a system

Virtualization stacks

Executing code on the client

A note about the exercises

The Puppet master and Puppet agents

Ansible

PalletOps

Deploying with Chef

Deploying with SaltStack

Salt versus Ansible versus Puppet versus PalletOps execution models

Vagrant

Deploying with Docker

Comparison tables

Cloud solutions

AWS

Azure

Summary

8. Monitoring the Code

Nagios

Munin

Ganglia

Graphite

Log handling

Client-side logging libraries

The ELK stack

Summary

9. Issue Tracking

What are issue trackers used for?

Some examples of workflows and issues

What do we need from an issue tracker?

Problems with issue tracker proliferation

All the trackers

Bugzilla

Trac

Redmine

The GitLab issue tracker

Jira

Summary

10. The Internet of Things and DevOps

Introducing the IoT and DevOps

The future of the IoT according to the market

Machine-to-machine communication

IoT deployment affects software architecture

IoT deployment security

Okay, but what about DevOps and the IoT again?

A hands-on lab with an IoT device for DevOps

Summary

II. Module 2

1. Basic Command Line Tools

Introduction

Controlling network interfaces

Getting ready

How to do it…

See also

Monitoring network details with the IP command

Getting ready

How to do it…

Monitoring connections using the ss command

Getting ready

How to do it…

Gathering basic OS statistics

Getting ready

How to do it…

Viewing historical resource usage with SAR

Getting ready

How to do it…

Installing and configuring a Git client

Getting ready

How to do it…

Creating an SSH key for Git

Getting ready

How to do it

How it works…

Using ssh-copy-id to copy keys

Getting ready

How to do it…

How it works…

See also

Creating a new Git repository

Getting ready

How to do it…

How it works…

See also

Cloning an existing Git repository

Getting ready

How to do it…

How it works…

See also

Checking changes into a Git repository

Getting ready

How to do it…

How it works…

See also

Pushing changes to a Git remote

Getting ready

How to do it…

How it works…

See also

Creating a Git branch

Getting ready

How to do it…

How it works…

See also

2. Ad Hoc Tasks with Ansible

Introduction

Installing an Ansible control node on Ubuntu

Getting ready

How to do it…

See also

Installing an Ansible control node on CentOS

Getting ready

How to do it…

See also

Creating an Ansible inventory

Getting ready

How to do it…

See also

Using the raw module to install python-simplejson

Getting ready

How to do it…

See also

Installing packages with Ansible

Getting ready

How to do it...

See also

Restarting services using Ansible

Getting ready

How to do it…

See also

Executing freeform commands with Ansible

Getting ready

How to do it…

Managing users with Ansible

Getting ready

How to do it…

See also

Managing SSH keys with Ansible

Getting ready

How to do it...

See also

3. Automatic Host builds

Introduction

Creating an Apt mirror using aptly

Getting ready

How to do it…

See also

Automated installation using PXE boot and a Preseed file

Getting ready

How to do it…

See also

Automating post-installation tasks

Getting ready

How to do it…

4. Virtualization with VMware ESXi

Introduction

Installing ESXi

Getting ready

How to do it…

Installing and using the vSphere Client

Getting ready

How to do it….

Allowing SSH access to ESXi

Getting ready

How to do it…

Creating a new guest

Getting ready

How to do it...

Allocating resources to a guest

Getting ready

How to do it…

Using the ESXi command line to start, stop, and destroy guests

Getting ready

How to do it…

Managing command-line snapshots

Getting ready

How to do it…

Tuning the host for guest performance

Getting ready

How to do it…

See also

5. Automation with Ansible

Introduction

Installing Ansible

Getting ready

How to do it…

See also

Creating a scaffold Playbook

Getting ready

How to do it

Creating a common role

Getting ready

How to do it…

See also

Creating a webserver using Ansible and Nginx

Getting ready

How to do it…

See also

Creating an application server role using Tomcat and Ansible

Getting ready

How to do it…

See also

Installing MySQL using Ansible

Getting ready

How to do it…

See also

Installing and managing HAProxy with Ansible

Getting ready

How to do it…

See also

Using ServerSpec to test your Playbook

Getting ready

How to do it…

See also

6. Containerization with Docker

Introduction

Installing Docker

Getting ready

How to do it…

See also

Pulling an image from the public Docker registry

Getting ready

How to do it…

See also

Performing basic Docker operations

Getting ready

How to do it…

See also

Running a container interactively

Getting ready

How to do it…

See also

Creating a Dockerfile

Getting ready

How to do it…

See also

Running a container in detached mode

Getting ready

How to do it…

See also

Saving and restoring a container

Getting ready

How to do it…

See also

Using the host only network

Getting ready

How to do it…

See also

Running a private Docker registry

Getting ready

How to do it

See also

Managing images with a private registry

Getting ready

How to do it…

Pushing images

Pulling images

See also

7. Using Jenkins for Continuous Deployment

Introduction

Installing Jenkins

Getting ready

How to do it…

See also…

Installing the Git plugin

Getting ready

How to do it…

See also

Installing a Jenkins slave

Getting ready

How to do it…

See also

Creating your first Jenkins job

Getting ready

How to do it…

See also

Building Docker containers using Jenkins

Getting ready

How to do it…

Deploying a Java application to Tomcat with zero downtime using Ansible

Getting ready

How to do it…

See also

8. Metric Collection with InfluxDB

Introduction

Installing InfluxDB

Getting ready

How to do it…

See also

Creating a new InfluxDB database

Getting ready

How to do it…

See also

Logging events with the InfluxDB REST API

Getting ready

How to do it…

See also

Gathering host statistics with Telegraf

Getting ready

How to do it…

See also

Exploring data with the InfluxDB data explorer

Getting ready

How to do it…

See also

Installing Grafana

Getting ready…

How to do it…

See also

Creating dashboards with Grafana

Getting ready

How to do it…

See also

9. Log Management

Introduction

Centralizing logs with Syslog

Getting ready

How to do it…

See also

Using syslog templates

Getting ready

How to do it…

See also

Managing log rotation with the Logrotate utility

Getting ready

How to do it…

See also

Installing ElasticSearch, Logstash, and Kibana

Getting ready

How to do it…

See also

Importing logs into Elasticsearch with Logstash

Getting ready

How to do it…

See also

Using Kibana queries to explore data

Getting ready

How to do it…

See also

Using Kibana queries to examine data

Getting ready

How to do it…

See also

10. Monitoring with Sensu

Introduction

Installing a Sensu server

Getting ready

How to do it…

See also

Installing a Sensu client

Getting ready

How to do it…

See also

Installing check prerequisites

Getting ready

How to do it…

Finding community checks

Getting ready

How to do it…

See also

Adding a DNS check

Getting ready

How to do it…

See also

Adding a disk check

Getting ready

How to do it…

See also

Adding a RAM check

Getting ready

How to do it…

See also

Adding a process check

Getting ready…

How to do it…

See also

Adding a CPU check

Getting ready

How to do it…

See also

Creating e-mail alerts

Getting ready

How to do it…

See also

Creating SMS alerts

Getting ready

How to do it…

See also

Using Ansible to install Sensu

Getting ready

How to do it…

See also

11. IAAS with Amazon AWS

Introduction

Signing up for AWS

Getting ready

How to do it…

See also

Setting up IAM

Getting ready

How to do it…

See also

Creating your first security group

Getting ready

How to do it…

See also

Creating your first EC2 host

Getting ready

How to do it…

See also

Using Elastic Load Balancers

Getting ready

How to do it…

See also

Managing DNS with route53

Getting ready…

How to do it…

See also

Using Ansible to create EC2 hosts

Getting ready

How to do it…

See also

12. Application Performance Monitoring with New Relic

Introduction

Signing up for a New Relic account

Getting ready

How to do it…

See also

Installing the New Relic Java agent

Getting ready

How to do it…

See also

Using the performance overview

Getting ready

How to do it…

See also

Locating performance bottlenecks with Transaction Traces

Getting ready

How to do it…

See also

Observing database performance with New Relic

Getting ready

How to do it…

See also

Release performance monitoring with New Relic

Getting ready

How to do it…

See also

Server Monitoring with New Relic

Getting ready

How to do it…

See also

III. Module 3

1. Evolution of a Software House

A brief history of ACME systems

ACME systems version 1.0

Software delivery process flow version 1.0

ACME systems version 2.0

Software delivery process flow version 2.0

A few brave men and women

ACME systems version 3.0

Software delivery process flow version 3.0

ACME systems version 4.0

The evolution in a nutshell

Summary

2. No Pain, No Gain

Elephant in the room

Defining the rules

Including (almost) everyone

Identifying the key people

Too many cooks

Openness, transparency, and honesty

Location, location, location

It's all happy-clappy management waffle – isn't it?

The great elephant disclosure

Value stream mapping

Summary

3. Plan of Attack

Setting and communicating the goal and vision

Standardizing vocabulary and language

A business change project in its own right

The merits of a dedicated team

Who to include

The importance of evangelism

Courage and determination

Understanding the cost

Seeking advice from others

Summary

4. Culture and Behaviors

All roads lead to culture

An open, honest, and safe environment

Openness and honesty

Courageous dialogue

The physical environment

Encouraging and embracing collaboration

Fostering innovation and accountability at grass roots

The blame culture

Blame slow, learn quickly

Building trust-based relationships across organizational boundaries

Rewarding good behaviors and success

The odd few

Recognizing dev and ops teams are incentivized can have an impact

Embracing change and reducing risk

Changing people's perceptions with pudding

Being transparent

Summary

5. Approaches, Tools, and Techniques

Engineering best practice

Source control

Small, frequent, and simple changes

Never break your consumer

Open and honest peer-working practices

Fail fast and often

Automated builds and tests

Continuous Integration

Using the same binary across all environments

How many environments are enough?

Developing against a production-like environment

CD tooling

Automated provisioning

No-downtime deployments

The cloud

Monitoring

When a simple manual process is also an effective tool

Summary

6. Hurdles Along the Way

What are the potential issues you need to look out for?

Dissenters in the ranks

No news is no news

The anti-agile brigade

The transition curve

The outsiders

Corporate guidelines, red tape, and standards

Geographically diverse teams

Failure during evolution

Processes that are not repeatable

Recruitment

Summary

7. Vital Measurements

Measuring effective engineering best practice

Simple quality metrics

Code complexity

Unit test coverage

Commit rates

Adherence to coding rules and standards

Where to start and why bother?

Measuring the real world

Measuring the stability of the environments

Incorporating automated tests

Combining automated tests and system monitoring

Real-time monitoring of the software itself

Monitoring utopia

Effectiveness of CD and DevOps

Impact of CD and DevOps

Measuring your culture

Summary

8. Are We There Yet?

Reflect on where you are now

Streaming

A victim of your own success

[P]lan, [D]o, [C]heck, [A]djust

Exit stage left

Rest on your laurels (not)

Summary

9. The Future is Bright

Expanding your horizon

Reactive performance and load testing

Reducing feature flag complexity

Easing A/B testing

Security patching and saving your bacon

Order out of chaos monkey

End user self-service

CD and DevOps and the mobile world

Expanding beyond software delivery

What about me?

What have you learned?

Summary

A. Some Useful Information

Tools

People

Recommended reading

B. Where Am I on the Evolutionary Scale?

C. Retrospective Games

The timeline game

StoStaKee

D. Vital Measurements Expanded

Code complexity – some science

Code versus comments

Embedding monitoring into your software

A. Bibliography

Index