Implementing Splunk: Big Data Reporting and Development for Operational Intellig电子书

Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

Table of Contents

Implementing Splunk: Big Data Reporting and Development for Operational Intelligence

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers and more

Why Subscribe?

Free Access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. The Splunk Interface

Logging in to Splunk

The Home app

The top bar

Search app

Data generator

The Summary view

Search

Actions

Timeline

The field picker

Fields

Search results

Options

Events viewer

Using the time picker

Using the field picker

Using Manager

Summary

2. Understanding Search

Using search terms effectively

Boolean and grouping operators

Clicking to modify your search

Event segmentation

Field widgets

Time

Using fields to search

Using the field picker

Using wildcards efficiently

Only trailing wildcards are efficient

Wildcards are tested last

Supplementing wildcards in fields

All about time

How Splunk parses time

How Splunk stores time

How Splunk displays time

How time zones are determined and why it matters

Different ways to search against time

Specifying time in-line in your search

_indextime versus _time

Making searches faster

Sharing results with others

Saving searches for reuse

Creating alerts from searches

Schedule

Actions

Summary

3. Tables, Charts, and Fields

About the pipe symbol

Using top to show common field values

Controlling the output of top

Using stats to aggregate values

Using chart to turn data

Using timechart to show values over time

timechart options

Working with fields

A regular expression primer

Commands that create fields

eval

rex

Extracting loglevel

Using the Extract Fields interface

Using rex to prototype a field

Using the admin interface to build a field

Indexed fields versus extracted fields

Indexed field case 1 – rare instances of a common term

Indexed field case 2 – splitting words

Indexed field case 3 – application from source

Indexed field case 4 – slow requests

Indexed field case 5 – unneeded work

Summary

4. Simple XML Dashboards

The purpose of dashboards

Using wizards to build dashboards

Scheduling the generation of dashboards

Editing the XML directly

UI Examples app

Building forms

Creating a form from a dashboard

Driving multiple panels from one form

Post-processing search results

Post-processing limitations

Panel 1

Panel 2

Panel 3

Final XML

Summary

5. Advanced Search Examples

Using subsearches to find loosely related events

Subsearch

Subsearch caveats

Nested subsearches

Using transaction

Using transaction to determine the session length

Calculating the aggregate of transaction statistics

Combining subsearches with transaction

Determining concurrency

Using transaction with concurrency

Using concurrency to estimate server load

Calculating concurrency with a by clause

Calculating events per slice of time

Using timechart

Calculating average requests per minute

Calculating average events per minute, per hour

Rebuilding top

Summary

6. Extending Search

Using tags to simplify search

Using event types to categorize results

Using lookups to enrich data

Defining a lookup table file

Defining a lookup definition

Defining an automatic lookup

Troubleshooting lookups

Using macros to reuse logic

Creating a simple macro

Creating a macro with arguments

Using eval to build a macro

Creating workflow actions

Running a new search using values from an event

Linking to an external site

Building a workflow action to show field context

Building the context workflow action

Building the context macro

Using external commands

Extracting values from XML

xmlkv

XPath

Using Google to generate results

Summary

7. Working with Apps

Defining an app

Included apps

Installing apps

Installing apps from Splunkbase

Using Geo Location Lookup Script

Using Google Maps

Installing apps from a file

Building your first app

Editing navigation

Customizing the appearance of your app

Customizing the launcher icon

Using custom CSS

Using custom HTML

Custom HTML in a simple dashboard

Using ServerSideInclude in a complex dashboard

Object permissions

How permissions affect navigation

How permissions affect other objects

Correcting permission problems

App directory structure

Adding your app to Splunkbase

Preparing your app

Confirming sharing settings

Cleaning up our directories

Packaging your app

Uploading your app

Summary

8. Building Advanced Dashboards

Reasons for working with advanced XML

Reasons for not working with advanced XML

Development process

Advanced XML structure

Converting simple XML to advanced XML

Module logic flow

Understanding layoutPanel

Panel placement

Reusing a query

Using intentions

stringreplace

addterm

Creating a custom drilldown

Building a drilldown to a custom query

Building a drilldown to another panel

Building a drilldown to multiple panels using HiddenPostProcess

Third-party add-ons

Google Maps

Sideview Utils

The Sideview Search module

Linking views with Sideview

Sideview URLLoader

Sideview forms

Summary

9. Summary Indexes and CSV Files

Understanding summary indexes

Creating a summary index

When to use a summary index

When to not use a summary index

Populating summary indexes with saved searches

Using summary index events in a query

Using sistats, sitop, and sitimechart

How latency affects summary queries

How and when to backfill summary data

Using fill_summary_index.py to backfill

Using collect to produce custom summary indexes

Reducing summary index size

Using eval and rex to define grouping fields

Using a lookup with wildcards

Using event types to group results

Calculating top for a large time frame

Storing raw events in a summary index

Using CSV files to store transient data

Pre-populating a dropdown

Creating a running calculation for a day

Summary

10. Configuring Splunk

Locating Splunk configuration files

The structure of a Splunk configuration file

Configuration merging logic

Merging order

Merging order outside of search

Merging order when searching

Configuration merging logic

Configuration merging example 1

Configuration merging example 2

Configuration merging example 3

Configuration merging example 4 (search)

Using btool

An overview of Splunk .conf files

props.conf

Common attributes

Search-time attributes

Index-time attributes

Parse-time attributes

Input time attributes

Stanza types

Priorities inside a type

Attributes with class

inputs.conf

Common input attributes

Files as inputs

Using patterns to select rolled logs

Using blacklist and whitelist

Selecting files recursively

Following symbolic links

Setting the value of host from source

Ignoring old data at installation

When to use crcSalt

Destructively indexing files

Network inputs

Native Windows inputs

Scripts as inputs

transforms.conf

Creating indexed fields

Creating a loglevel field

Creating a session field from source

Creating a "tag" field

Creating host categorization fields

Modifying metadata fields

Overriding host

Overriding source

Overriding sourcetype

Routing events to a different index

Lookup definitions

Wildcard lookups

CIDR wildcard lookups

Using time in lookups

Using REPORT

Creating multivalue fields

Creating dynamic fields

Chaining transforms

Dropping events

fields.conf

outputs.conf

indexes.conf

authorize.conf

savedsearches.conf

times.conf

commands.conf

web.conf

User interface resources

Views and navigation

Appserver resources

Metadata

Summary

11. Advanced Deployments

Planning your installation

Splunk instance types

Splunk forwarders

Splunk indexer

Splunk search

Common data sources

Monitoring logs on servers

Monitoring logs on a shared drive

Consuming logs in batch

Receiving syslog events

Receiving events directly on the Splunk indexer

Using a native syslog receiver

Receiving syslog with a Splunk forwarder

Consuming logs from a database

Using scripts to gather data

Sizing indexers

Planning redundancy

Indexer load balancing

Understanding typical outages

Working with multiple indexes

Directory structure of an index

When to create more indexes

Testing data

Differing longevity

Differing permissions

Using more indexes to increase performance

The lifecycle of a bucket

Sizing an index

Using volumes to manage multiple indexes

Deploying the Splunk binary

Deploying from a tar file

Deploying using msiexec

Adding a base configuration

Configuring Splunk to launch at boot

Using apps to organize configuration

Separate configurations by purpose

Configuration distribution

Using your own deployment system

Using Splunk deployment server

Step 1 – Deciding where your deployment server will run

Step 2 – Defining your deploymentclient.conf configuration

Step 3 – Defining our machine types and locations

Step 4 – Normalizing our configurations into apps appropriately

Step 5 – Mapping these apps to deployment clients in serverclass.conf

Step 6 – Restarting the deployment server

Step 7 – Installing deploymentclient.conf

Using LDAP for authentication

Using Single Sign On

Load balancers and Splunk

web

splunktcp

deployment server

Multiple search heads

Summary

12. Extending Splunk

Writing a scripted input to gather data

Capturing script output with no date

Capturing script output as a single event

Making a long-running scripted input

Using Splunk from the command line

Querying Splunk via REST

Writing commands

When not to write a command

When to write a command

Configuring commands

Adding fields

Manipulating data

Transforming data

Generating data

Writing a scripted lookup to enrich data

Writing an event renderer

Using specific fields

Table of fields based on field value

Pretty print XML

Writing a scripted alert action to process results

Summary

Index