PySpark Training in Newcastle

This instructor-led live training in Newcastle covers the Stratio platform, focusing on the Rocket and Intelligence modules with PySpark. Participants will master data ingestion, transformation, and advanced analytics, gaining practical skills in loops, UDFs, and enterprise data workflows.

In this instructor-led, live training in Newcastle, participants will learn how to utilise Python and Spark in tandem to analyse big data, supported by hands-on exercises.

By the end of this training, participants will be able to:

• Learn how to use Spark with Python to analyse Big Data.
• Work on exercises that mirror real-world scenarios.
• Use different tools and techniques for big data analysis using PySpark.

This live, instructor-led training (available online or on-site) is tailored for data engineers, analysts, and professionals aiming to utilise Databricks and PySpark to build scalable data pipelines and migrate established SQL workflows.

This training provides a practical introduction to constructing scalable data processing and Machine Learning workflows using PySpark. It covers the operation of Apache Spark within modern Big Data ecosystems and the efficient processing of large datasets through distributed computing principles.

This three-day practical course focuses on building and optimising efficient data-processing workloads using PySpark, Pandas and Polars in Kubernetes-based environments.

Participants will develop a practical understanding of how Spark applications execute on Kubernetes and how application-level configuration decisions influence performance, scalability, resource consumption and cost. The course covers key optimisation areas including executor sizing, memory allocation, dynamic allocation, partitioning strategies, shuffle behaviour, small-file problems and efficient Parquet processing.

The course also addresses common challenges when working with Pandas, including memory limitations and out-of-memory failures, and introduces Polars as a high-performance alternative for selected data-processing workloads. Through hands-on exercises, participants will diagnose performance and memory issues, compare different configuration strategies and apply optimisation techniques to realistic ETL and machine learning scenarios.

The emphasis throughout the course is on practical decision-making: understanding how to identify bottlenecks, select the appropriate tool, configure Spark efficiently and balance performance with infrastructure resource consumption and cost.