Learning Hunk电子书

Learning Hunk

Table of Contents

Learning Hunk

Credits

About the Authors

About the Reviewer

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

1. Meet Hunk

Big data analytics

The big problem

The elegant solution

Supporting SPL

Intermediate results

Getting to know Hunk

Splunk versus Hunk

Hunk architecture

Connecting to Hadoop

Advance Hunk deployment

Native versus virtual indexes

Native indexes

Virtual index

External result provider

Computation models

Data streaming

Data reporting

Mixed mode

Hunk security

One Hunk user to one Hadoop user

Many Hunk users to one Hadoop user

Hunk user(s) to the same Hadoop user with different queues

Setting up Hadoop

Starting and using a virtual machine with CDH5

SSH user

MySQL

Starting the VM and cluster in VirtualBox

Big data use case

Importing data from RDBMS to Hadoop using Sqoop

Telecommunications – SMS, Call, and Internet dataset from dandelion.eu

Milano grid map

CDR aggregated data import process

Periodical data import from MySQL using Sqoop and Oozie

Problems to solve

Summary

2. Explore Hadoop Data with Hunk

Setting up Hunk

Extracting Hunk to a VM

Setting up Hunk variables and configuration files

Running Hunk for the first time

Setting up a data provider and virtual index for CDR data

Setting up a connection to Hadoop

Setting up a virtual index for data stored in Hadoop

Accessing data through a virtual index

Exploring data

Creating reports

The top five browsers report

Top referrers

Site errors report

Creating alerts

Creating a dashboard

Controlling security with Hunk

The default Hadoop security

One Hunk user to one Hadoop user

Summary

3. Meeting Hunk Features

Knowledge objects

Field aliases

Calculated fields

Field extractions

Tags

Event type

Workflow actions

Macros

Data model

Add auto-extracting fields

Adding GeoIP attributes

Other ways to add attributes

Introducing Pivot

Summary

4. Adding Speed to Reports

Big data performance issues

Hunk report acceleration

Creating a virtual index

Streaming mode

Creating an acceleration search

What's going on in Hadoop?

Report acceleration summaries

Reviewing summary details

Managing report accelerations

Hunk accelerations limits

Summary

5. Customizing Hunk

What we are going to do with the Splunk SDK

Supported languages

Solving problems

REST API

The implementation plan

The conclusion

Dashboard customization using Splunk Web Framework

Functionality

A description of time-series aggregated CDR data

Source data

Creating a virtual index for Milano CDR

Creating a virtual index for the Milano grid

Creating a virtual index using sample data

Implementation

Querying the visualization

Downloading the application

Custom Google Maps

Page layout

Linear gradients and bins for the activity value

Custom map components

Other components

The final result

Summary

6. Discovering Hunk Integration Apps

What is Mongo?

Installation

Installing the Mongo app

Mongo provider

Creating a virtual index

Inputting data from the recommendation engine backend

Data schemas

Data mechanics

Counting by shop in a single collection

Counting events in all collections

Counting events in shops for observed days

Summary

7. Exploring Data in the Cloud

An introduction to Amazon EMR and S3

Amazon EMR

Setting up an Amazon EMR cluster

Amazon S3

S3 as a data provider for Hunk

The advantages of EMR and S3

Integrating Hunk with EMR and S3

Method 1: BYOL

Setting up the Hunk AMI

Adding a license

Configuring the data provider

Configuring a virtual index

Setting up a provider and virtual index in the configuration file

Exploring data

Method 2: Hunk–hourly pricing

Provisioning a Hunk instance using the Cloud formation template

Provisioning a Hunk instance using the EC2 Console

Converting Hunk from an hourly rate to a license

Summary

Index