The Agenda

Agenda Home
SQLBits 2024 runs from Tuesday 19th – Saturday 23rd March.
Analytics

Microsoft Data Integration Pipelines – The Fundamentals to Level 300

Description

In this full day of training, we will start with the very basics of data integration before building up the fundamental skills and artifacts needed to orchestrate your data platform end-to-end. We will do this using Azure Data Factory pipelines, not exclusively, but as the basis for our learning journey. The maturity of the Data Factory resource offers a set of good foundations to understand the technical capabilities needed for orchestration, before applying our knowledge to other tools such as Microsoft Fabric and Azure Synapse Analytics.

Through a blend of theory, demonstration, and practical labs we will explore the components needed to orchestration cloud-based data processing workloads from source system ingestion, through to data model delivery. Focusing on the control flow plane, with supplementary options for building out low code data flow packages available as part of our integration pipelines.

Start the day knowing nothing about data integration pipelines, or as an experienced SQL Server SSIS developer, and leave with the knowledge and resources to apply these skills to your role as a data professional working with Microsoft cloud native tools.

Learning Objectives


• How cloud native data integration resources have evolved over time.
• What the basic data pipeline artifacts are.
• What the common data movement deployment patterns are.
• How to build complex, high dynamic control flows.
• How to massively scale out executions and handle parallel orchestration workloads.
• Best practices for the deployment of orchestration resources.

Previous Experience

Any data workload handling experience, including familiarity with SQL Server SSIS would be a great starting point. But not necessary as we’ll cover the fundamentals.

Tech Covered

Functions, Data Lake, Modern Analytics, Analytics, Fabric, Data Factory