The Agenda

Agenda Home
SQLBits 2024 runs from Tuesday 19th – Saturday 23rd March.

Open Hack: From ingestion to consumption

Description

This workshop is designed to get hands on experience with a Modern Data Solution. During this day, the participants will be wrangling data inside the Azure Data Platform, going from ingestion to consumption. At the end of the day, the participants will have the required skillset to build their own modern data warehouse using the full power of the Azure stack.

The workshop covers 5 modules.

Module 1 will take a deeper look into the various Azure storage options that can be used as the landing zone for the data. The architecture needs to fulfill a set of customer requirements. Based on these requirements and the guidance of the experts, a data architecture will be designed to realize in the following modules.

Module 2 will focus on loading the data to your architecture, and again making sure you are compliant with all the customer requirements. We will need to apply the newest standards when it comes DevOps and create a full flow of data from source to destination using Azure components like Azure Data Factory, Azure Storage, Azure SQLDB/DW, Databricks, ... .

Module 3 will require data rendering skills where you will cover the consumption part of your data. A report must be created based on the data you’ve processed in the previous modules. The goal is to create valuable insights to support the customer in their decision making on critical business processes. Here we will be using PowerBI

Module 4 will introduce a new data stream to the architecture. In this module you will combine the batch data with live stream data and make sure this data is persisted and can be queried in real-time. In a modern world, data can’t be transformed into insights fast enough. Learn everything about creating a real time data pipeline. (Components Virtual IOT Device, IOT hub, Stream Analytics, Databricks)

Module 5 will cover AI using algorithms to detect and solve a real-world data science challenge. Based on the newest techniques in the azure suite, you will start to analyze the data and apply machine learning models on your data architecture. This is the next level of data maturity where even decision making is automatized.  (ML.net, Cognitive services)

For this workshop, you will be working in small groups of 3-5 people. There will be experts present during the workshop who can help you with all the questions you might have, and they will also guide you in choosing one of the possible solutions for the scenario. This ensures a high level of interactivity to have an excellent knowledge transfer between both your team and the experts. The winner of the day is determined based on their progress, quality and team spirit. It’s not about winning; participating will already give you the tools to become a modern data expert.

You will need to bring your own laptop with SSMS, Visual Studio and Visual Studio code installed. An Azure access promo code will be provided to you for the duration of the workshop.

Learning Objectives

Previous Experience

Tech Covered

Deploying, Securing, Cloud, data warehousing, Synapse Analytics, ML, Azure SQL Database, Power BI, Azure ML, Stream Analytics, Event Hub, Azure Data Factory, Azure Synapse Analytics, Azure Storage, Modern Analytics, Data transformation and integration, AI and data science, Move to the cloud, Azure Data Engineer, Steam Analytics, Open Hack, Developing, Managing, Monitoring, Optimising, Data Bricks