Course Code

spmllib

Duration

35 hours (usually 5 days including breaks)

Requirements

Knowledge of one of the following:

Java
Scala
Python
SparkR.

Overview

MLlib is Spark’s machine learning (ML) library. Its goal is to make practical machine learning scalable and easy. It consists of common learning algorithms and utilities, including classification, regression, clustering, collaborative filtering, dimensionality reduction, as well as lower-level optimization primitives and higher-level pipeline APIs.

It divides into two packages:

spark.mllib contains the original API built on top of RDDs.
spark.ml provides higher-level API built on top of DataFrames for constructing ML pipelines.

Audience

This course is directed at engineers and developers seeking to utilize a built in Machine Library for Apache Spark

Course Outline

spark.mllib: data types, algorithms, and utilities

Data types
Basic statistics
- summary statistics
- correlations
- stratified sampling
- hypothesis testing
- streaming significance testing
- random data generation
Classification and regression
- linear models (SVMs, logistic regression, linear regression)
- naive Bayes
- decision trees
- ensembles of trees (Random Forests and Gradient-Boosted Trees)
- isotonic regression
Collaborative filtering
- alternating least squares (ALS)
Clustering
- k-means
- Gaussian mixture
- power iteration clustering (PIC)
- latent Dirichlet allocation (LDA)
- bisecting k-means
- streaming k-means
Dimensionality reduction
- singular value decomposition (SVD)
- principal component analysis (PCA)
Feature extraction and transformation
Frequent pattern mining
- FP-growth
- association rules
- PrefixSpan
Evaluation metrics
PMML model export
Optimization (developer)
- stochastic gradient descent
- limited-memory BFGS (L-BFGS)

spark.ml: high-level APIs for ML pipelines

Overview: estimators, transformers and pipelines
Extracting, transforming and selecting features
Classification and regression
Clustering
Advanced topics

Testimonials

★★★★★

This is one of the best hands-on with exercises programming courses I have ever taken.

Laura Kahn

Course:Artificial Intelligence - the most applied stuff - Data Analysis + Distributed AI + NLP

Richard is very calm and methodical, with an analytic insight - exactly the qualities needed to present this sort of course.

Kieran Mac Kenna

Course:Spark for Developers

We know a lot more about the whole environment.

John Kidd

Course:Spark for Developers

The trainer made the class interesting and entertaining which helps quite a bit with all day training.

Ryan Speelman

Course:Spark for Developers

Ernesto did a great job explaining the high level concepts of using Spark and its various modules.

Michael Nemerouf

Course:Spark for Developers

Richard was very willing to digress when we wanted to ask semi-related questions about things not on the syllabus. Explanations were clear and he was up front about caveats in any advice he gave us.

ARM Limited

Course:Spark for Developers

The VM I liked very much The Teacher was very knowledgeable regarding the topic as well as other topics, he was very nice and friendly I liked the facility in Dubai.

Safar Alqahtani - Elm Information Security

Course:Big Data Analytics in Health

Small group (4 trainees) and we could progress together. Also the trainer could so help everybody.

ICE International Copyright Enterprise Germany GmbH

Course:Spark for Developers

Ajay was very friendly, helpful and also knowledgable about the topic he was discussing.

Biniam Guulay - ICE International Copyright Enterprise Germany GmbH

Course:Spark for Developers

The lab exercises. Applying the theory from the first day in subsequent days.

Dell

Course:A Practical Introduction to Stream Processing

This is one of the best quality online training I have ever taken in my 13 year career. Keep up the great work!.

Course:Artificial Intelligence - the most applied stuff - Data Analysis + Distributed AI + NLP

I think the trainer had an excellent style of combining humor and real life stories to make the subjects at hand very approachable. I would highly recommend this professor in the future.

Course:Spark for Developers

The trainer was passionate and well-known what he said I appreciate his help and answers all our questions and suggested cases.

Course:A Practical Introduction to Stream Processing