English | 2024 | ISBN: 978-1804616024 | 272 Pages | EPUB | 10 MB
Ensure your data pipelines are healthy and promote data observability in your teams with this essential hands-on guide
Key Features
- Learn how to monitor your data pipelines in a scalable way
- Use real-life use cases and projects to practice implementing data observability
- Build trust in your pipelines among data producers and consumers alike
In the information age, data is critically important. Every organization needs to manage its data effectively to ensure accuracy and to prevent its data pipelines from breaking. In these fast moving times of data engineering, how can you keep on top of this?
Data Observability for Data Engineering has the answer. Data observability is a union of techniques and methods that allow you to monitor and validate the health of your data, and this practical guide will show you how to implement it successfully in your organization.
We begin by explaining what data observability is, how it builds on data quality monitoring, and why it is essential from data engineering perspective. Once you’re familiar with the techniques and elements of data observability, you’ll get hands-on with a practical Python project to reinforce what you’ve learned.
At the end of the book, we provide some use cases and projects for you to experiment with, by which time you will be perfectly placed to implement Data Observability in your organization and never worry again about the quality of your data pipelines to ease the mind of data engineers.
What you will learn
- Monitor data pipelines proactively in a scalable way
- Implement a data observability approach in the pipelines
- Collect and analyze key metrics through coding examples
- Apply monkey patching in a Python module
- Manage the costs and risks of your data pipeline
- Understand the main techniques to collect observability metrics
- Implement analytics pipeline monitoring techniques in production
- Build a statistic engine continuously
Resolve the captcha to access the links!