HBase Administration Cookbook电子书

HBase Administration Cookbook

Table of Contents

HBase Administration Cookbook

Credits

About the Author

Acknowledgement

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers and more

Why Subscribe?

Free Access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Setting Up HBase Cluster

Introduction

Quick start

Getting ready

How to do it...

How it works...

Getting ready on Amazon EC2

Getting ready

How to do it...

How it works...

Setting up Hadoop

Getting ready

How to do it...

How it works...

Setting up ZooKeeper

Getting ready

How to do it...

How it works...

There's more...

Changing the kernel settings

Getting ready

How to do it...

How it works...

See also

Setting up HBase

Getting ready

How to do it...

How it works...

Basic Hadoop/ZooKeeper/HBase configurations

How to do it...

How it works...

See also

Setting up multiple High Availability (HA) masters

Getting ready

How to do it...

Install and configure Heartbeat and Pacemaker

Create and install a NameNode resource agent

Configure highly available NameNode

Start DataNode, HBase cluster, and backup HBase master

How it works...

There's more...

2. Data Migration

Introduction

Importing data from MySQL via single client

Getting ready

How to do it...

How it works...

Importing data from TSV files using the bulk load tool

Getting ready

How to do it...

How it works...

There's more...

Writing your own MapReduce job to import data

Getting ready

How to do it...

How it works...

There's more...

Generating HFile files in MapReduce

Important configurations affecting data migration

See also

Precreating regions before moving data into HBase

Getting ready

How to do it...

How it works...

See also

3. Using Administration Tools

Introduction

HBase Master web UI

Getting ready

How to do it...

How it works...

Using HBase Shell to manage tables

Getting ready

How to do it...

How it works...

There's more...

Using HBase Shell to access data in HBase

Getting ready

How to do it...

How it works...

See also

Using HBase Shell to manage the cluster

Getting ready

How to do it...

How it works...

See also

Executing Java methods from HBase Shell

Getting ready

How to do it...

How it works...

There's more...

Row counter

Getting ready

How to do it...

How it works...

There's more...

WAL tool—manually splitting and dumping WALs

Getting ready

How to do it...

How it works...

See also

HFile tool—viewing textualized HFile content

Getting ready

How to do it...

How it works...

There's more...

HBase hbck—checking the consistency of an HBase cluster

Getting ready

How to do it...

How it works...

See also

Hive on HBase—querying HBase using a SQL-like language

Getting ready

How to do it...

How it works...

4. Backing Up and Restoring HBase Data

Introduction

Full shutdown backup using distcp

Getting ready

How to do it...

How it works...

Using CopyTable to copy data from one table to another

Getting ready

How to do it...

How it works...

Exporting an HBase table to dump files on HDFS

Getting ready

How to do it...

How it works...

There's more...

See also

Restoring HBase data by importing dump files from HDFS

Getting ready

How to do it...

How it works...

There's more...

See also

Backing up NameNode metadata

Getting ready

How to do it...

How it works...

There's more...

Backing up region starting keys

Getting ready

How to do it...

How it works...

See also

Cluster replication

Getting ready

How to do it...

How it works...

There's more...

5. Monitoring and Diagnosis

Introduction

Showing the disk utilization of HBase tables

Getting ready

How to do it...

How it works...

There's more...

Setting up Ganglia to monitor an HBase cluster

Getting ready

How to do it...

How it works...

There's more...

See also

OpenTSDB—using HBase to monitor an HBase cluster

Getting ready

How to do it...

How it works...

There's more...

Setting up Nagios to monitor HBase processes

Getting ready

How to do it...

How it works...

There's more...

Using Nagios to check Hadoop/HBase logs

Getting ready

How to do it...

How it works...

There's more...

See also

Simple scripts to report the status of the cluster

Getting ready

How to do it...

How it works...

There's more...

See also

Hot region—write diagnosis

Getting ready

How to do it...

How it works...

There's more...

See also

6. Maintenance and Security

Introduction

Enabling HBase RPC DEBUG-level logging

Getting ready

How to do it...

How it works...

There's more...

Graceful node decommissioning

Getting ready

How to do it...

How it works...

There's more...

See also

Adding nodes to the cluster

Getting ready

How to do it...

How it works...

There's more...

Rolling restart

Getting ready

How to do it...

How it works...

There's more...

Simple script for managing HBase processes

Getting ready

How to do it...

How it works...

Simple script for making deployment easier

Getting ready

How to do it...

How it works...

There's more...

Kerberos authentication for Hadoop and HBase

Getting ready

How to do it...

How it works...

There's more...

See also

Configuring HDFS security with Kerberos

Getting ready

How to do it...

How it works...

There's more...

HBase security configuration

Getting ready

How to do it...

How it works...

There's more...

7. Troubleshooting

Introduction

Troubleshooting tools

Getting ready

How to do it...

How it works...

See also

Handling the XceiverCount error

Getting ready

How to do it...

How it works...

Handling the "too many open files" error

Getting ready

How to do it...

How it works...

There's more...

See also

Handling the "unable to create new native thread" error

Getting ready

How to do it...

How it works...

There's more...

See also

Handling the "HBase ignores HDFS client configuration" issue

Getting ready

How to do it...

How it works...

Handling the ZooKeeper client connection error

Getting ready

How to do it...

How it works...

There's more...

Handling the ZooKeeper session expired error

Getting ready

How to do it...

How it works...

See also

Handling the HBase startup error on EC2

Getting ready

How to do it...

How it works...

There's more...

See also

8. Basic Performance Tuning

Introduction

Setting up Hadoop to spread disk I/O

Getting ready

How to do it...

How it works...

There's more...

Using network topology script to make Hadoop rack-aware

Getting ready

How to do it...

How it works...

Mounting disks with noatime and nodiratime

Getting ready

How to do it...

How it works...

There's more...

Setting vm.swappiness to 0 to avoid swap

Getting ready

How it works...

See also

Java GC and HBase heap settings

Getting ready

How to do it...

How it works...

There's more...

See also

Using compression

Getting ready

How to do it...

How it works...

There's more...

Managing compactions

Getting ready

How to do it...

How it works...

There's more...

Managing a region split

Getting ready

How to do it...

How it works...

There's more...

See also

9. Advanced Configurations and Tuning

Introduction

Benchmarking HBase cluster with YCSB

Getting ready

How to do it...

How it works...

There's more...

Increasing region server handler count

Getting ready

How to do it...

How it works...

See also

Precreating regions using your own algorithm

Getting ready

How to do it...

How it works...

There's more...

See also

Avoiding update blocking on write-heavy clusters

Getting ready

How to do it...

How it works...

See also

Tuning memory size for MemStores

Getting ready

How to do it...

How it works...

There's more...

See also

Client-side tuning for low latency systems

Getting ready

How to do it...

How it works...

There's more...

Configuring block cache for column families

Getting ready

How to do it...

How it works...

There's more...

See also

Increasing block cache size on read-heavy clusters

Getting ready

How to do it...

How it works...

See also

Client side scanner setting

Getting ready

How to do it...

How it works...

There's more...

See also

Tuning block size to improve seek performance

Getting ready

How to do it...

How it works...

There's more...

See also

Enabling Bloom Filter to improve the overall throughput

Getting ready

How to do it...

How it works...

There's more...

Index