1

Module 1 : Introduction

• About This Course
• About Cloudera

Session 1

2 Hrs

2

Module 2: Introduction to Hadoop and the Hadoop Ecosystem

• Hadoop
• Data Storage and Ingest
• Data Processing
• Data Analysis and Exploration
• Other Ecosystem Tools
• Introduction to the Homework Labs
• Homework Labs: Setup and General Notes

Session 2

2 Hrs

3

Hadoop Architecture and HDFS

• Hadoop
• Data Storage and Ingest
• Data Processing
• Data Analysis and Exploration
• Other Ecosystem Tools
• Introduction to the Homework Labs
• Homework Labs: Setup and General Notes

Session 3

2 Hrs

4

Importing and Modeling Structured Data,Importing Relational Data with Apache Sqoop

• Sqoop Overview
• Basic Imports and Exports
• Limiting Results
• Improving Sqoop’s Performance
• Sqoop 2
• Homework Labs: Import Data from MySQL Using Sqoop

Session 4

2 Hrs

5

Introduction to Impala and Hive

• Introduction to Impala and Hive
• Why Use Impala and Hive?
• Querying Data With Impala and Hive
• Comparing Hive and Impala to Traditional Databases

Session 5

2 Hrs

6

Modeling and Managing Data with Impala and Hive

• Data Storage Overview
• Creating Databases and Tables
• Loading Data into Tables
• HCatalog
• Impala Metadata Caching
• Homework Labs: Create and Populate Tables in Impala or Hive

Session 6

2 Hrs

7

Data Formats

• File Formats
• Avro Schemas
• Avro Schema Evolution
• Using Avro with Impala, Hive and Sqoop
• Using Parquet with Impala, Hive and Sqoop
• Compression
• Homework Labs: Select a Format for a Data File

Session 7

2 Hrs

8

Data File Partitioning

• Partitioning Overview
• Partitioning in Impala and Hive
• Conclusion
• Homework Labs:Partition Data in Impala or Hive

Session 8

2 Hrs

9

Module 4: Ingesting Streaming Data

• What is Apache Flume?
• Basic Flume Architecture
• Flume Sources
• Flume Sinks
• Flume Channels
• Flume Configuration
• Homework Labs: Collect Web Server Logs with Flume

Session 9

2 Hrs

10

Module 5: Distributed Data Processing with SparkSpark Basics

• What is Apache Spark?
• Using the Spark Shell
• RDDs (Resilient Distributed Datasets)
• Functional Programming in Spark
• Homework Labs:
• View the Spark Documentation
• Explore RDDs Using the Spark Shell
• Use RDDs to Transform a Dataset

Session 10

2 Hrs

11

Working with RDDs in Spark

• Creating RDDs
• Other General RDD Operations
• Homework Labs:Process Data Files with Spark

Session 11

2 Hrs

12

Aggregating Data with Pair RDDs

• Key?Value Pair RDDs
• Map?Reduce
• Other Pair RDD Operations
• Homework Labs:Use Pair RDDs to Join Two Datasets

Session 12

2 Hrs

13

Writing and Deploying Spark Applications

• Spark Applications vs. Spark Shell
• Creating the SparkContext
• Building a Spark Application (Scala and Java)
• Running a Spark Application
• The Spark Application Web UI
• Homework Labs:
• Write and Run a Spark Application
• Configuring Spark Properties
• Logging
• b>Homework Labs:Configure a Spark Application

Session 13

2 Hrs

14

Parallel Processing in Spark

• Review: Spark on a Cluster
• RDD Partitions
• Partitioning of File?based RDDs
• HDFS and Data Locality
• Executing Parallel Operations
• Stages and Tasks
• Homework Labs:View Jobs and Stages in the Spark Application UI

Session 14

2 Hrs

15

Spark RDD Persistence

• RDD Lineage
• RDD Persistence Overview
• Distributed Persistence
• Homework Labs:Persist an RDD

Session 15

2 Hrs

16

Common Patterns in Spark Data Processing

• Common Spark Use Cases
• Iterative Algorithms in Spark
• Graph Processing and Analysis
• Machine Learning
• Example: k?means
• Homework Labs: Iterative Processing in Spark
• Optional Homework Lab: Partition Data Files Using Spark

Session 16

2 Hrs

17

Spark SQL and DataFrames

• Spark SQL and the SQL Context
• Creating DataFrames
• Transforming and Querying DataFrames
• Saving DataFrames
• DataFrames and RDDs
• Comparing Spark SQL, Impala and Hive?on?Spark
• Homework Labs:Use Spark SQL for ETL

Session 17

2 Hrs

18

Conclusion and Project Discussion

Session 18

2 Hrs