Spark Training

Spark Training

 

Invite friends

Contact Us

Page Views : 137

About The Event

for more details contact us at +91-720-720-9005 ( Whats app ) or www.dwbiadda.com

 

Introduction to Big Data

What is Big Data

Challenges with Big Data

Batch Vs. Real Time Big Data Analytics

 

Introduction of Spark

What is Spark

Why Spark

Who Uses Spark

Brief History of Spark

Storage Layers for Spark

Unified Stack of Spark

    1. Spark Core
    2. Spark Sql
    3. Spark Streaming
    4. MLlib
    5. GraphX

 

Spark Architecture explanation

    1. Master Slave architecture
    2. Spark Driver
    3. Workers
    4. Executors

 

Working with Spark in different Scala

Scala

    1. How to use 'spark-shell'
    2. Practical examples on spark in Scala

 

 

Creating the Spark Context

Configuring Spark Context with SparkConf

Caching Overview

Distributed Persistence

Combine scala and java seamlessly

Deploying Applications with spark-submit

Verify spark jobs in Spark Web UI

Installing maven

Building a Spark Project with maven

Running Spark Project with maven

Resilient Distributed Dataset (RDD)

What is RDD

Creating RDDs

RDD Operations

    1. Transformations
    2. Actions
    3. Lazy Evaluation

Passing Functions to Spark

 

 

 

Working with Key/Value Pairs

Creating Pair RDDs

Transformations on Pair RDDs

    1. Aggregations
    2. Grouping Data
    3. Joins
    4. Sorting Data

Data Partitioning

    1. Determining an RDD’s Partitioner
    2. Custom Partitioners

 

Loading and Saving Your Data

File Formats

    1. Text, Json, csv, tsv, Object files
    2. Hadoop Input and Output Formats

Loading Data using RDD

Saving Data using RDD

MapReduce and Pair RDD Operations

Scala and Hadoop Integrations

 

Broadcast and Accumulators

 

Accumulators

    1. Introduction to Accumulators
    2. Practical Examples on Accumulators
    3. Creating Custom Accumulators

 

Broadcast variables

 

    1. Introduction to Broadcast variables
    2. Practical Examples on Broadcast variables
    3. Optimizing Broadcasts

 

Apache Spark SQL

Hive and Spark SQL Architecture explanation

Working with Spark SQL DataFrames

Using Spark SQL Context

Practical examples on Spark SQL

Integrating hive and Spark SQL

Creating & Using SQL Hive Context

Hive Queries through Spark

Processing Text, JSON and Parquet Files using in Spark

Spark SQL Performance Tuning Options

JDBC/ODBC Server

 

Apache Spark Streaming

Spark Streaming Architecture explanation

Transformations

Output Operations

Streaming UI explanation

Performance Considerations

Practical examples on Spark Streaming

 

Apache Spark MLlib

Machine Learning Basics

Machine Learning Algorithms

 

    1. Classification
    2. Clustering
    3. Collaborative Filtering

         Performance Considerations

Practical examples on Spark MLlib

 

 

Apache Storm

Introduction to Apache Storm

Apache Storm Architecture explanation

Practical Examples on Apache Storm

 

Apache Kafka

Introduction to Apache Kafka

Apache Kafka Architecture explanation

Practical Examples on Apache Kafka

 

Introduction of Scala

What is Scala?

Why Scala?

Advantages of Scala?

Using the Scala REPL(Read Evaluate print loop)

What is Type Inference

Interoperability between Scala and Java

 

Scala using Command Line

Installing Java & Scala

Interactive Scala

Writing Scala s

Compiling Scala Programs

 

Basics of Scala

Defining Variables

Defining Functions

String Interpolation

IDE for Scala

 

Scala Type Less, Do More

Semicolons

Variable Declarations

Method Declarations

Type Inference

Immutability

Reserved Words

Operators

Precedence Rules

Literals

Options

Arrays, Lists, Ranges, Tuples

 

Expressions and Conditionals

If expressions

If-Else expressions

Match Expressions

For Loops

While Loops

Do-While Loops

Conditional Operators

Enumerations

Pattern Matching

Using try, catch, and finally Clauses

 

Functional Programming in Scala

What is Functional Programming?

Functional Literals and Closures

Recursions

Tail Calls

Currying

Functional Data Structures

Implicit Function Parameters

Call by Name, Call by Value

 

Object-Oriented Programming in Scala

Class and Object Basics

Value Classes

Parent Types

Constructors in Scala

Fields in Classes

 

 

Nested Types

 

    1. Hands on installation Spark and it's relative software's in your laptop.
    2. Discussing about Spark & Scala interview questions daily base.
    3. Resume preparation with Project's based on your experience.