Course Outline
Introduction:
- Apache Spark within the Hadoop ecosystem
- Brief overview of Python and Scala
Foundations (theory):
- Architecture
- RDD (Resilient Distributed Dataset)
- Transformations and Actions
- Stages, Tasks, and Dependencies
Exploring fundamentals in a Databricks environment (hands-on workshop):
- Practical exercises using the RDD API
- Core action and transformation functions
- Working with PairRDDs
- Join operations
- Caching strategies
- Practical exercises using the DataFrame API
- Spark SQL
- DataFrame operations: select, filter, group, sort
- UDFs (User Defined Functions)
- An introduction to the DataSet API
- Streaming concepts
Understanding deployment in an AWS environment (hands-on workshop):
- Core concepts of AWS Glue
- Key differences between AWS EMR and AWS Glue
- Example jobs in both environments
- Evaluating the advantages and disadvantages
Additional topics:
- Introduction to Apache Airflow orchestration
Requirements
Programming skills (preferably in Python or Scala)
Foundational knowledge of SQL
Custom Corporate Training
Training solutions designed exclusively for businesses.
- Customised Content: We adapt the syllabus and practical exercises to the real goals and needs of your project.
- Flexible Schedule: Dates and times adapted to your team's agenda.
- Format: Online (live), In-company (at your offices), or Hybrid.
Price per private group, online live training, starting from £4800 + VAT*
Contact us for an exact quote and to hear our latest promotions
Testimonials (3)
Having hands on session / assignments
Poornima Chenthamarakshan - Intelligent Medical Objects
Course - Apache Spark in the Cloud
1. Right balance between high level concepts and technical details. 2. Andras is very knowledgeable about his teaching. 3. Exercise
Steven Wu - Intelligent Medical Objects
Course - Apache Spark in the Cloud
Get to learn spark streaming , databricks and aws redshift