Big Data Forensics – Learning Hadoop Investigations电子书

Big Data Forensics – Learning Hadoop Investigations

Table of Contents

Big Data Forensics – Learning Hadoop Investigations

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the color images of this book

Errata

Piracy

Questions

1. Starting Out with Forensic Investigations and Big Data

An overview of computer forensics

The forensic process

Identification

Collection

Analysis

Presentation

Other investigation considerations

Equipment

Evidence management

Investigator training and certification

The post-investigation process

What is Big Data?

The four Vs of Big Data

Big Data architecture and concepts

Big Data forensics

Metadata preservation

Collection methods

Collection verification

Summary

2. Understanding Hadoop Internals and Architecture

The Hadoop architecture

The components of Hadoop

The Hadoop Distributed File System

The Hadoop configuration files

Hadoop daemons

Hadoop data analysis tools

Hive

HBase

Pig

Managing files in Hadoop

File permissions

Trash

Log files

File compression and splitting

Hadoop SequenceFile

The Hadoop archive files

Data serialization

Packaged jobs and JAR files

The Hadoop forensic evidence ecosystem

Running Hadoop

LightHadoop

Amazon Web Services

Loading Hadoop data

Importing sample data for testing

Summary

3. Identifying Big Data Evidence

Identifying evidence

Locating sources of data

Compiling data requirements

Reviewing the system architecture

Interviewing staff and reviewing the documentation

Assessing data viability

Identifying data sources in noncooperative situations

Data collection requirements

Data source identification

Structured and unstructured data

Data collection types

In-house or third-party collection

The types of data to request

The data collection request

An investigator-led collection

The chain of custody documentation

Summary

4. Collecting Hadoop Distributed File System Data

Forensically collecting a cluster system

Physical versus remote collections

HDFS collections through the host operating system

Imaging the host operating system

Imaging a mounted HDFS partition

Targeted collection from a Hadoop client

The Hadoop shell command collection

Collecting HDFS files

HDFS targeted data collection

Hadoop Offline Image and Edits Viewers

Collection via Sqoop

Other HDFS collection approaches

Summary

5. Collecting Hadoop Application Data

Application collection approaches

Backups

Query extractions

Script extractions

Software extractions

Validating application collections

Collecting Hive evidence

Loading Hive data

Identifying Hive evidence

Hive backup collection

Hive query collection

Hive query control totals

Hive metadata and log collection

The Hive script collection

Collecting HBase evidence

Loading HBase data

Identifying HBase evidence

The HBase backup collection

The HBase query collection

HBase collection via scripts

HBase control totals

HBase metadata and log collection

Collecting other Hadoop application data and non-Hadoop data

Summary

6. Performing Hadoop Distributed File System Analysis

The forensic analysis process

Forensic analysis goals

Forensic analysis concepts

The challenges of forensic analysis

Anti-forensic techniques

Data encryption

Analysis preparation

Analysis

Keyword searching and file and data carving

Bulk Extractor

Autopsy

Metadata analysis

File activity timeline analysis

Other metadata analysis

The analysis of deleted files

HDFS data extraction

Hex editors

Cluster reconstruction

Configuration file analysis

Linux configuration files

Hadoop configuration files

Hadoop application configuration files

Log file analysis

Summary

7. Analyzing Hadoop Application Data

Preparing the analysis environment

Pre-analysis steps

Loading data

Preload data transformations

Data surveying

Transforming data

Transforming nonrelational data

Analyzing data

The analysis approach

Types of investigation

Analysis techniques

Isolating known facts and events

Grouping and clustering

Histograms

The time series analysis

Measuring change over time

Anomaly detection

Rule-based analysis

Duplication analysis

Benford's law

Aggregation analysis

Plotting outliers on a timeline

Analyzing disparate data sets

Keyword searching

Validating the findings

Documenting the findings

Summary

8. Presenting Forensic Findings

Types of reports

Sample reports

Internal investigation report

Affidavit and declaration

Expert report

Developing the report

Explaining the process

Showing the findings

Using exhibits or appendices

Testimony and other presentations

Summary

Index