Building Google Cloud Platform Solutions电子书

Title Page

Copyright and Credits

Building Google Cloud Platform Solutions

About Packt

Why subscribe?

Packt.com

Contributors

About the authors

Packt is searching for authors like you

Preface

Who this book is for

What this book covers

To get the most out of this book

Download the example code files

Conventions used

Get in touch

Reviews

Why GCP?

The public cloud landscape

Amazon Web Services

Microsoft Azure

Google Cloud Platform

Standing on the shoulders of giants

A world-class global presence

Choosing your own adventure

Leading the way for big data

The Open Cloud and innovation

Dedication to customer success

Bottom-up security

In good company

Summary

The Google Cloud Console

Getting started – Google Cloud projects

Architectural role of Google Cloud projects

Creating a project

Free trials on GCP

The Google Cloud Console

Understanding the Cloud Console dashboard

The Google Cloud Shell

Launching the Cloud Shell

Supporting multiple sessions

Features and integrations

File management

Web Preview

The Cloud Shell Code Editor

Opening in Cloud Shell

Trying it out

Installing additional tools

Boost mode

Repairing the Cloud Shell

Other tools

Mobile apps

Developer tool integrations

Summary

APIs, CLIs, IAM, and Billing

Google Cloud APIs

Managing APIs

Google APIs Explorer

Trying out the APIs Explorer

The Google Cloud SDK

Installing the Google Cloud SDK

The gcloud command-line tool

The basics of gcloud

Command groups

Root commands

Global flags

Initializing the Google Cloud SDK

Authentication

Managing your Google Cloud SDK

Updating and rollbacks

Alpha and beta channels

Configurations in the Google Cloud SDK

Modifying configuration properties

Multiple configurations

Other command-line tools

bq

gsutil

kubectl

Automating tasks with gcloud

Modifying output and behavior

Formatting attributes

Formatting projections

Filtering

Google Cloud IAM

How IAM works

IAM roles

The structure of IAM policies

Organization-level policies

Project-level policies

Resource-level policies

Cross-project access

Managing IAM

Service accounts

Billing on Google Cloud

Billing accounts

Billing accounts and IAM

Budgets and billing alerts

Google Cloud Platform Pricing Calculator

Creating an estimate

Summary

Google App Engine

Compute services on the GCP

Google Compute Engine

Google Kubernetes Engine (GKE)

Google App Engine

Google Cloud Functions

General considerations

Google App Engine

Features and benefits

Developer velocity

Visibility

Scalability

Simple integrations

Structure of a Google App Engine application

Architecture of an App Engine solution

Microservices

Batch work and task queues

App Engine locations

IAM on the Google App Engine

App Engine service accounts

The standard and flexible environments

Standard environment

Flexible environment

Setting up the App Engine

The App Engine standard environment

Language support

Developing for the App Engine standard environment

The Python runtime

WSGI and CGI

Getting started

The App Engine development server

The Go runtime

Running multiple services locally

The Java 8 runtime

Deploying App Engine standard services

Deployment behavior

Splitting network traffic

Instance classes

Pricing in the standard environment

Spending limits

The App Engine flexible environment

Benefits of the flexible environment

More control over the infrastructure

Application portability

Language support

Developing for the flexible environment

Deploying App Engine flexible apps

Container technologies

Google Container Builder

Google Container Registry

Custom runtimes

Building custom runtime services

Deploying a service to the flexible environment

Pricing in the flexible environment

App Engine resources and integrations

Task queues

Push and pull queues

Push queues

Named queues

Pull queues

Creating tasks

Structuring tasks queues

Scheduled tasks

Deploying a cron definition

Trying the App Engine cron service

Scaling App Engine services

Autoscaling

Basic and manual scaling

Externalizing configuration and managing secrets

Application configuration files

Compute Engine metadata server

Runtime Configurator

Cloud Key Management Service (KMS)

General considerations

Networking and security

The App Engine firewall

Cloud Endpoints

Google Cloud IAP

Virtual private networks

Summary

Google Kubernetes Engine

Google Kubernetes Engine

When to choose GKE

GKE or App Engine Flex

Creating and maintaining a GKE cluster

Node pools

Multi-zonal and regional clusters

Container Registry

Deploying workloads to GKE

Rolling updates

Rolling back updates

Scaling deployments

Manually scaling deployments

Automatically scaling deployments

Exposing GKE Services

Exposing services within a cluster

Exposing services to external traffic

Managing secrets with GKE

Creating/Storing secrets

Using secrets

Billing

Summary

Google Cloud Functions

Functions as a Service

Google Cloud Functions

Advantages of Cloud Functions

Price

Scalability

Developer velocity

Considerations when using Cloud Functions

Invoking Cloud Functions

HTTP functions

Processing HTTP requests

Background functions

Cloud Pub/Sub functions

Cloud Storage functions

Background function retries and termination

Developing Cloud Functions

Using the Cloud Console

Local development

Debugging functions

Deploying Cloud Functions

Deploying from a local machine

Deploying from a source repository

Integrating with other Google services

IAM and billing

Cloud Functions and IAM

Frameworks and tooling

Summary

Google Compute Engine

Understanding Compute Engine

IaaS

Infrastructure as Code (IaC)

More than virtual machines

When to use Compute Engine

A straightforward migration path

Host anything

Building a robust global presence

Long running and resource intensive processes

Security and compliance

Virtual machines on Google Compute Engine (GCE)

Machine types

Standard machine types

High-memory machine types

Mega-memory machine types

High-CPU machine types

Shared-core machine types

Custom machine types

Extended memory

Other resources

Disk storage

GPUs

Images

Public images

Premium images

Community images

Container images

Managing Compute Engine instances

Creating instances

Remote access

SSH access

SCP access

Remote Desktop Protocol (RDP) access

Metadata server

Default metadata

Project-wide metadata

Instance-specific metadata

Setting and removing metadata

Querying metadata from within instances

Trying it out

Modifying API responses

Startup and shutdown scripts

Startup scripts

Shutdown Scripts

Windows machines

Updates and patches

Availability policies

Maintenance behavior

Restart behavior

Relocating an instance

Storage solutions

Persistent disks

Standard and solid-state drive (SSD) persistent disks

Persistent disk performance

Boot disks

Managing persistent disks

Persistent disk snapshots

Local SSDs

Creating scalable solutions with GCE

Custom images

Creating images from a persistent disk

Copying an image

Creating images from snapshots

Golden images

Security concerns

Managed instance group (MIG)

Instance templates

Creating MIGs

Built for resilience

Autoscaling

Autohealing

Change management

Performing a rolling update

IAM and service accounts

Administrative operations

General roles

Compute resource roles

Network and security resource roles

Compute instance IAM

Pricing on GCE

Instance discounts

Preemptible instances

Committed use discounts

Sustained use discounts

Other resource costs

Always-free tier

Summary

NoSQL with Datastore and Bigtable

NoSQL solutions on GCP

NoSQL technologies

Google Cloud Datastore

When to use Datastore

Getting started

Datastore locations

Managing entities in the Cloud Console

Datastore core concepts

The structure of Datastore data

Entities, kinds, and properties

Data types

Entity identifiers

Namespaces

Ancestry paths and keys

Entity groups and consistency

Entity groups

Consistency and queries

Working with entities

Queries with GQL

Using GQL in the Cloud Console

Indexes

Single property indexes

Composite indexes

Datastore under the hood

The entities table

Key

Entity group

Kind

Properties

Custom indexes

Index tables

EntitiesByKind

EntitiesByProperty

EntitesByCompositeProperty and Custom Indexes

Datastore management and integrations

Administrative tasks

The Datastore Admin Console

gcloud operations

Integrations with other GCP services

App Engine standard environment

Other GCP services

Datastore pricing and IAM

Permissions in Datastore

Google Cloud Firestore

Comparison to Datastore

A promising future

Google Bigtable

Core concepts

Structure of Bigtable data

Columns and column families

Column families

Scalable and intelligent

Bigtable under the hood

Building on other Google technologies

Tablets and servers

Creating and managing clusters

Instances, clusters, and nodes

Development instances

Bigtable locations

Create a development cluster

Using gcloud

Scaling clusters

Promoting development clusters

Deleting a cluster

Interacting with data on Bigtable

The cbt command-line interface

The Bigtable HBase Client

Platform integrations

BigQuery external tables

Dataflow Bigtable IO

Bigtable pricing and IAM

Permissions in Bigtable

Summary

Relational Data with Cloud SQL and Cloud Spanner

Google Cloud SQL

Configuring Cloud SQL instances

Creating a Cloud SQL instance

Database engines

MySQL generations

Machine and storage types

Choosing a machine type

Configuring storage

Cloud SQL locations

When to use multiple instances

Connecting to Cloud SQL

Authorized networks

Connecting with gcloud

SSL support

Establishing an SSL Connection

The Cloud SQL Proxy

Setting up the Cloud SQL Proxy

Authenticating with the Cloud SQL Proxy

Trying it out

Managing Cloud SQL instances

Maintenance operations

Importing data to Cloud SQL

Exporting data to cloud storage

Backups and recovery

Trying it out

Point-in-time recovery

Updates

Database flags

Database flags and SLAs

Replicas and high availability

Read-only replicas

External replicas

High availability

Forcing a failover

Scaling Cloud SQL instances

Scaling Storage

Scaling compute

Alerting on resource pressure

Horizontal scaling

Migrating databases to Cloud SQL

Cloud SQL IAM and users

IAM policies

Database users

Default and system users

Additional users

Changing user passwords

Cloud SQL Proxy users

Cloud SQL pricing

Google Cloud Spanner

Instances and instance configurations

Regional configurations

Multi-region configurations

Nodes, databases, and tables

Creating a Cloud Spanner instance

Importing data into Cloud Spanner

Performing a simple query

Understanding Cloud Spanner

Cloud Spanner and CAP theorem

Maintaining consistency

TrueTime and linearization

Paxos groups

Read operations

Write operations

Transactions

Database design and optimizations

Query execution plans

Primary keys

Data collocation and interleaving

Secondary indexes and index directives

Cloud Spanner administration

Cloud Spanner IAM Roles

Cloud Spanner prices

Summary

Google Cloud Storage

GCS basics

Buckets

Bucket names

Domain-named buckets

The global bucket namespace

Objects

Object data

Object metadata

Virtual file structures

Using gsutil

Creating and using a bucket

Uploading files to GCS

Storage classes and locations

Regional and Multi-Regional Storage

Standard and durable reduced availability

Nearline and Coldline Storage

Cloud Storage locations

Nearline and Coldline Storage locations

Choosing the right storage class

Cloud Storage pricing

Bucket and object storage classes

Automating object management

Monitoring lifecycle events

Object versioning

Data governance in Cloud Storage

Cloud Storage IAM

ACLs

Limitations of concentric access control

Customer supplied encryption keys

Signed URLs

Capabilities and integrations

Integrating with Google Cloud Functions

Static web content and Backend Buckets

Summary

Stackdriver

Lessons from SRE

Monitoring and alerting

Preparation for this chapter

Stackdriver basics

Stackdriver and GCP projects

Creating and linking a Stackdriver account

Stackdriver Logging

Filtering and searching

Basic filtering

Advanced filtering

Exporting Stackdriver logs

Exporting to Cloud Storage

Exporting to BigQuery and Cloud Pub/Sub

Monitoring and alerting

The Stackdriver Monitoring console

Exploring Stackdriver metrics

Creating dashboards

Stackdriver alerting policies

Policy conditions

Creating an alerting policy

Notifications and documentation

Stackdriver incidents

Other types of metrics

Error reporting

Investigating errors

Stackdriver APM

Stackdriver Trace

Investigating application latency

Stackdriver Debugger

Debugging the todos services

Logpoints

Stackdriver Profiler

Summary

Change Management

Preparing for this chapter

Google Cloud Source Repositories

Google Cloud Deployment Manager

Declarative configuration management

Basic configurations

Resource types and properties

Deployments

Deploying a simple configuration

Deployment manifests

Updating deployments

Create and delete policies

Maintaining deployment state

Remediation

Templates

Creating a template

Other template features

Cloud Launcher and Deployment Manager

Runtime Configurator

Watchers

Waiters

Google Cloud Container services

Google Container Registry – GCR

Container Builder

Build triggers

Continuous deployment in Google Cloud

Summary

GCP Networking for Developers

Networking fundamentals

Virtual private networks

Subnetworks

Configuring VPC networks

Networks and compute resources

Firewall rules

Components of a firewall rule

Action

Direction

Target

Source or destination

Protocol and port

Priority

Securing networks with firewall rules

Routes

IP addresses

Internal and external IP addresses

Ephemeral and static IP addresses

Global IP addresses

Google load balancers

Network load balancers

Target pools

Forwarding rules

Health checks

Failover ratio and backup pools

Creating a TCP network load balancer

Internal load balancing

Global load balancers

Components of global load balancers

Backend services

Target proxies

Global forwarding rules

SSL and TCP proxies

HTTP(S) load balancers

Autoscaling load balanced resources

Google Cloud DNS

Access control and API management

Google Cloud Endpoints

Services

API providers

Access and discovery

Identity-Aware Proxy

Cloud Armor

Summary

Messaging with Pub/Sub and IoT Core

Google Cloud Pub/Sub

Topics and subscriptions

Push and pull message delivery

Pull subscriptions

Push subscriptions

Choosing a subscription model

Message acknowledgment

Nacking messages

Designing for resilience

Message loss

Processing failures

Duplicate messages

Out-of-order messages

Google Cloud IoT Core

Device management and registries

Device authentication and security

Consuming device data

Summary

Integrating with Big Data Solutions on GCP

Big data and Google Cloud Platform

Cloud Dataflow

Evolution of data processing at Google

Pipelines

Collections

Transformations

Element-wise transforms

Aggregate transforms

Composite transforms

Sources and sinks

Creating and executing pipelines

Executing pipelines locally

Executing pipelines on Cloud Dataflow

Executing streaming pipelines

Pipeline templates

Google provided pipeline templates

Managing Cloud Dataflow jobs

Google BigQuery

How BigQuery executes queries

Integrating with BigQuery

BigQuery as a Cloud Dataflow Sink

Batch loading files from Cloud Storage

Streaming inserts

Exploring BigQuery data

Summary

Compute

Introduction

Hosting a Node.js application on Google Compute Engine

Getting ready

How to do it...

Running the application on the development machine

Deploying the application on GCP

Moving the code to Google Source Repositories

Creating the start up script

Creating and configuring a GCE instance

Hosting the Node.js application on Google App Engine

Getting ready

How to do it...

Hosting a Node.js application on Kubernetes Engine

Getting ready

How to do it...

Creating a cluster on GKE to host the containers

Containerizing the KeystoneJS application

Creating a replicated deployment for the application and MongoDB

Hosting an application on Google Cloud Functions

Getting ready

How to do it...

Hosting a highly scalable application on Google Compute Engine

Getting ready

How to do it...

How it works...

Storage and Databases

Introduction

Hosting a static application using Google Cloud Storage

Getting ready

How to do it...

There's more...

Image resizing using Google Cloud Storage and Cloud Functions

Getting ready

How to do it...

How it works...

Migrating a MariaDB to Cloud Spanner

Getting ready

How to do it...

Loading temperature data to Cloud Bigtable and BigQuery-ing it

Getting ready

How to do it...

Storage metadata in Google Cloud Datastore

Getting ready

How to do it...

There's more...

Networking

Introduction

Network management using a data and a management network

Getting ready

How to do it...

There's more...

Content-based load balancing

Getting ready

How to do it...

VPC network peering between two networks

Getting ready

How to do it...

VPN connection between two networks

Getting ready

How to do it...

CDN setup for a static website

Getting ready

How to do it...

Security

Introduction

Scanning for vulnerabilities using Cloud Security Scanner

Getting ready

How to do it...

How it works...

Using Identity-Aware Proxy on App Engine

Getting ready

How to do it...

Securing financial data using Cloud KMS

Getting ready

How to do it...

Using Data Loss Prevention API

Getting ready

How to do it...

Creating IAM custom roles

Getting ready

How to do it...

Creating service accounts

Getting ready

How to do it...

Automatic recon and defense

Getting ready

How to do it...

Machine Learning and Big Data

Introduction

Uploading data to the Google BigQuery table

Getting ready

How to do it...

There's more...

Translating text to a target language

Getting ready

How to do it...

See also

Creating a Dataflow pipeline to store streaming data

Getting ready

How to do it...

Using the Vision API

Getting ready

How to do it...

Using the Google Cloud Speech API

Getting ready

How to do it...

Using the Cloud IoT Core

Getting ready

How to do it...

Management Tools

Introduction

Creating alerts on specific events

Getting ready

How to do it...

Monitoring a webserver using Stackdriver Monitoring

Getting ready

How to do it...

How it works...

Creating tailor-made instances using Deployment Manager

Getting ready

How to do it...

Error Reporting on a Python Flask application

Getting ready

How to do it...

Viewing application state using Stackdriver Debugger

Getting ready

How to do it...

Best Practices

Introduction

Creating a custom VM using Terraform

Getting ready

How to do it...

Monitoring a GCP account using Security Monkey

Getting ready

How to do it...

Creating instance backups

Getting ready

How to do it...

Simulating VM migration

Getting ready

How to do it...

Creating a golden image using Packer

Getting ready

How to do it...

Other Books You May Enjoy

Leave a review - let other readers know what you think