Monitoring Hadoop电子书

Monitoring Hadoop

Table of Contents

Monitoring Hadoop

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Errata

Piracy

Questions

1. Introduction to Monitoring

The need for monitoring

The monitoring tools available in the market

Nagios

Nagios architecture

Prerequisites for installing and configuring Nagios

Prerequisites

Installing Nagios

Web interface configuration

Nagios plugins

Verification

Configuration files

Setting up monitoring for clients

Ganglia

Ganglia components

Ganglia installation

System logging

Collection

Transportation

Storage

Alerting and analysis

The syslogd and rsyslogd daemons

Summary

2. Hadoop Daemons and Services

Hadoop daemons

NameNode

DataNode and TaskTracker

Secondary NameNode

JobTracker and YARN daemons

The communication between daemons

YARN framework

Common issues faced on Hadoop cluster

Host-level checks

Nagios server

Configuring Hadoop nodes for monitoring

Summary

3. Hadoop Logging

The need for logging events

System logging

Logging levels

Logging in Hadoop

Hadoop logs

Hadoop log level

Hadoop audit

Summary

4. HDFS Checks

HDFS overview

Nagios master configuration

The Nagios client configuration

Summary

5. MapReduce Checks

MapReduce overview

MapReduce control commands

MapReduce health checks

Nagios master configuration

Nagios client configuration

Summary

6. Hadoop Metrics and Visualization Using Ganglia

Hadoop metrics

Metrics contexts

Named contexts

Metrics system design

Metrics configuration

Configuring Metrics2

Exploring the metrics contexts

Hadoop Ganglia integration

Hadoop metrics configuration for Ganglia

Setting up Ganglia nodes

Hadoop configuration

Metrics1

Metrics2

Ganglia graphs

Metrics APIs

The org.apache.hadoop.metrics package

The org.apache.hadoop.metrics2 package

Summary

7. Hive, HBase, and Monitoring Best Practices

Hive monitoring

Hive metrics

HBase monitoring

HBase Nagios monitoring

HBase metrics

Monitoring best practices

The Filter class

Nagios and Ganglia best practices

Summary

Index