MEAPSA

Use Spark Notebooks in an Azure Synapse Pipeline

Apache Spark provides data engineers with a scalable, distributed data processing platform, which can be integrated into an Azure Synapse Analytics pipeline.

Data Engineer

Synapse Analytics

Module Objectives

In this module, you will learn how to:

Describe notebook and pipeline integration.
Use a Synapse notebook activity in a pipeline.
Use parameters with a notebook activity.

Units

Introduction min
Understand Synapse Notebooks and Pipelines min
Use a Synapse notebook activity in a pipeline min
Use parameters in a notebook min
Exercise - Use an Apache Spark notebook in a pipeline min
Knowledge check min
Summary min

Prerequisites

Before starting this module, you should have experience of using Apache Spark in Azure Synapse Analytics. Consider completing the Analyze data with Apache Spark in Azure Synapse Analytics module first: