Spark Training

for more details contact us at +91-720-720-9005 ( Whats app ) or www.dwbiadda.com

Introduction to Big Data

What is Big Data

Challenges with Big Data

Batch Vs. Real Time Big Data Analytics

Introduction of Spark

What is Spark

Why Spark

Who Uses Spark

Brief History of Spark

Storage Layers for Spark

Unified Stack of Spark

1. Spark Core
2. Spark Sql
3. Spark Streaming
4. MLlib
5. GraphX

Spark Architecture explanation

1. Master Slave architecture
2. Spark Driver
3. Workers
4. Executors

Working with Spark in different Scala

Scala

1. How to use 'spark-shell'
2. Practical examples on spark in Scala

Creating the Spark Context

Configuring Spark Context with SparkConf

Caching Overview

Distributed Persistence

Combine scala and java seamlessly

Deploying Applications with spark-submit

Verify spark jobs in Spark Web UI

Installing maven

Building a Spark Project with maven

Running Spark Project with maven

Resilient Distributed Dataset (RDD)

What is RDD

Creating RDDs

RDD Operations

1. Transformations
2. Actions
3. Lazy Evaluation

Passing Functions to Spark

Working with Key/Value Pairs

Creating Pair RDDs

Transformations on Pair RDDs

1. Aggregations
2. Grouping Data
3. Joins
4. Sorting Data

Data Partitioning

1. Determining an RDD’s Partitioner
2. Custom Partitioners

Loading and Saving Your Data

File Formats

1. Text, Json, csv, tsv, Object files
2. Hadoop Input and Output Formats

Loading Data using RDD

Saving Data using RDD

MapReduce and Pair RDD Operations

Scala and Hadoop Integrations

Broadcast and Accumulators

Accumulators

1. Introduction to Accumulators
2. Practical Examples on Accumulators
3. Creating Custom Accumulators

Broadcast variables

1. Introduction to Broadcast variables
2. Practical Examples on Broadcast variables
3. Optimizing Broadcasts

Apache Spark SQL

Hive and Spark SQL Architecture explanation

Working with Spark SQL DataFrames

Using Spark SQL Context

Practical examples on Spark SQL

Integrating hive and Spark SQL

Creating & Using SQL Hive Context

Hive Queries through Spark

Processing Text, JSON and Parquet Files using in Spark

Spark SQL Performance Tuning Options

JDBC/ODBC Server

Apache Spark Streaming

Spark Streaming Architecture explanation

Transformations

Output Operations

Streaming UI explanation

Performance Considerations

Practical examples on Spark Streaming

Apache Spark MLlib

Machine Learning Basics

Machine Learning Algorithms

1. Classification
2. Clustering
3. Collaborative Filtering

         Performance Considerations

Practical examples on Spark MLlib

Apache Storm

Introduction to Apache Storm

Apache Storm Architecture explanation

Practical Examples on Apache Storm

Apache Kafka

Introduction to Apache Kafka

Apache Kafka Architecture explanation

Practical Examples on Apache Kafka

Introduction of Scala

What is Scala?

Why Scala?

Advantages of Scala?

Using the Scala REPL(Read Evaluate print loop)

What is Type Inference

Interoperability between Scala and Java

Scala using Command Line

Installing Java & Scala

Interactive Scala

Writing Scala s

Compiling Scala Programs

Basics of Scala

Defining Variables

Defining Functions

String Interpolation

IDE for Scala

Scala Type Less, Do More

Semicolons

Variable Declarations

Method Declarations

Type Inference

Immutability

Reserved Words

Operators

Precedence Rules

Literals

Options

Arrays, Lists, Ranges, Tuples

Expressions and Conditionals

If expressions

If-Else expressions

Match Expressions

For Loops

While Loops

Do-While Loops

Conditional Operators

Enumerations

Pattern Matching

Using try, catch, and finally Clauses

Functional Programming in Scala

What is Functional Programming?

Functional Literals and Closures

Recursions

Tail Calls

Currying

Functional Data Structures

Implicit Function Parameters

Call by Name, Call by Value

Object-Oriented Programming in Scala

Class and Object Basics

Value Classes

Parent Types

Constructors in Scala

Fields in Classes

Nested Types

1. Hands on installation Spark and it's relative software's in your laptop.
2. Discussing about Spark & Scala interview questions daily base.
3. Resume preparation with Project's based on your experience.