万本电子书0元读

万本电子书0元读

顶部广告

Hadoop Backup and Recovery Solutions电子书

售       价:¥

37人正在读 | 0人评论 9.8

作       者:Gaurav Barot

出  版  社:Packt Publishing

出版时间:2015-07-28

字       数:217.6万

所属分类: 进口书 > 外文原版书 > 电脑/网络

温馨提示:此类商品不支持退换货,不支持下载打印

为你推荐

  • 读书简介
  • 目录
  • 累计评论(0条)
  • 读书简介
  • 目录
  • 累计评论(0条)
If you are a Hadoop administrator and you want to get a good grounding in how to back up large amounts of data and manage Hadoop clusters, then this book is for you.
目录展开

Hadoop Backup and Recovery Solutions

Table of Contents

Hadoop Backup and Recovery Solutions

Credits

About the Authors

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Knowing Hadoop and Clustering Basics

Understanding the need for Hadoop

Apache Hive

Apache Pig

Apache HBase

Apache HCatalog

Understanding HDFS design

Getting familiar with HDFS daemons

Scenario 1 – writing data to the HDFS cluster

Scenario 2 – reading data from the HDFS cluster

Understanding the basics of Hadoop cluster

Summary

2. Understanding Hadoop Backup and Recovery Needs

Understanding the backup and recovery philosophies

Replication of data using DistCp

Updating and overwriting using DistCp

The backup philosophy

Changes since the last backup

The rate of new data arrival

The size of the cluster

Priority of the datasets

Selecting the datasets or parts of datasets

The timelines of data backups

Reducing the window of possible data loss

Backup consistency

Avoiding invalid backups

The recovery philosophy

Knowing the necessity of backing up Hadoop

Determining backup areas – what should I back up?

Datasets

Block size – a large file divided into blocks

Replication factor

A list of all the blocks of a file

A list of DataNodes for each block – sorted by distance

The ACK package

The checksums

The number of under-replicated blocks

The secondary NameNode

Fixing the disk that has been corrupted or repairing it

Recovering the edit log

Recovering the state from the secondary NameNode

Active and passive nodes in second generation Hadoop

Hardware failure

Data corruption on disk

Disk/node failure

Rack failure

Software failure

Applications

Configurations

Is taking backup enough?

Understanding the disaster recovery principle

Knowing a disaster

The need for recovery

Understanding recovery areas

Summary

3. Determining Backup Strategies

Knowing the areas to be protected

Understanding the common failure types

Hardware failure

Host failure

Using commodity hardware

Hardware failures may lead to loss of data

User application failure

Software causing task failure

Failure of slow-running tasks

How Hadoop handles slow-running tasks

Speculative execution

Hadoop's handling of failing tasks

Task failure due to data

Data loss or corruption

No live node contains block errors

Bad data handling – through code

Hadoop's skip mode

Handling skip mode in Hadoop

Learning a way to define the backup strategy

Why do I need a strategy?

What should be considered in a strategy?

Filesystem check (fsck)

Filesystem balancer

Upgrading your Hadoop cluster

Designing network layout and rack awareness

Most important areas to consider while defining a backup strategy

Understanding the need for backing up Hive metadata

What is Hive?

Hive replication

Summary

4. Backing Up Hadoop

Data backup in Hadoop

Distributed copy

Architectural approach to backup

HBase

HBase history

HBase introduction

Understanding the HBase data model

Accessing HBase data

Approaches to backing up HBase

Snapshots

Operations involved in snapshots

Snapshot operation commands

HBase replication

Modes of replication

Export

The copy table

HTable API

Offline backup

Comparing backup options

Summary

5. Determining Recovery Strategy

Knowing the key considerations of recovery strategy

Disaster failure at data centers

How HDFS handles failures at data centers

Automatic failover configuration

How automatic failover configuration works

How to configure automatic failover

The transitionToActive and transitionToStandBy commands

Failover

The getServiceState command

The checkHealth command

How HBase handles failures at data centers

Restoring a point-in time copy for auditing

Restoring a data copy due to user error or accidental deletion

Defining recovery strategy

Centralized configuration

Monitoring

Alerting

Teeing versus copying

Summary

6. Recovering Hadoop Data

Failover to backup cluster

Installation and configuration

The user and group settings

Java installation

Password-less SSH configuration

ZooKeeper installation

Hadoop installation

The test installation of Hadoop

Hadoop configuration for an automatic failover

Preparing for the HA state in ZooKeeper

Formatting and starting NameNodes

Starting the ZKFC services

Starting DataNodes

Verifying an automatic failover

Importing a table or restoring a snapshot

Pointing the HBase root folder to the backup location

Locating and repairing corruptions

Recovering a drive from the working state

Lost files

The recovery of NameNode

What did we do just now?

Summary

7. Monitoring

Monitoring overview

Metrics of Hadoop

FileContext

GangliaContext

NullContextWithUpdateThread

CompositeContext

Java Management Extension

Monitoring node health

Hadoop host monitoring

Hadoop process monitoring

The HDFS checks

The MapReduce checks

Cluster monitoring

Managing the HDFS cluster

Logging

Log output written via log4j

Setting the log levels

Getting stack traces

Summary

8. Troubleshooting

Understanding troubleshooting approaches

Understanding common failure points

Human errors

Configuration issues

Hardware failures

Resource allocation issues

Identifying the root cause

Knowing issue resolution techniques

Summary

Index

累计评论(0条) 1个书友正在讨论这本书 发表评论

发表评论

发表评论,分享你的想法吧!

买过这本书的人还买过

读了这本书的人还在读

回顶部