SQLBits 2024

From DevOps for data to DataOps

In practice, DataOps is not as common for data & analytics as DevOps is for software engineering. For the latter, Development and Operations are jointly responsible for developing a system, deploying it and maintaining the system. With the aim of delivering faster, being more agile and creating maximum business value. This is where DataOps is the same as DevOps: the objective is similar. But ‘How’ we do this, differs considerably.
In practice, DataOps is not as common for data & analytics as DevOps is for software engineering. For the latter, Development and Operations are jointly responsible for developing a system, deploying it and maintaining the system. With the aim of delivering faster, being more agile and creating maximum business value. This is where DataOps is the same as DevOps: the objective is similar. But ‘How’ we do this, differs considerably.

Having the right data in the right place at the right time with the right quality, is becoming increasingly important for supporting business decisions, optimizing, automating and powering AI models. Just like with software development, you want to deliver new functionalities with premium quality much faster. You don’t want to make new data, new insights, new AI models available to the user every month, but when it is ready for deployment. That is what DataOps can achieve in theory. But in practice one faces serious challenges that make it a lot more difficult to effectuate the DataOps process in an organization. For example, how to deal with development sandboxes and representative test data across systems.

In this session Niels Naglé and Vincent Goris will show what DataOps is and that it is not just DevOps for data. They will discuss the unique challenges, solutions for these challenges and their lessons learned.

* How does DataOps relate to DevOps and what are the differences?
* A roadmap to implement DataOps in your organization
* The effect on your teams and organization
* The importance of Testing, Governance and DevOps
* The challenges and practical solutions.