- Continuous and disruptive database design updates while trying to have one design pattern to fit all use cases.
- Sub-par performance while loading, retrieving, and querying data.
- You want to shorten time-to-market of the projects aimed at data delivery and consumption.
- Unnecessarily complicated database design limits usability of the data and requires knowledge of specific data structures for their effective use.
- Evolve your data architecture. Data pipeline is an evolutionary break away from the enterprise data warehouse methodology.
- Avoid endless data projects. Building centralized all-in-one enterprise data warehouses takes forever to deliver a positive ROI.
- Facilitate data self-service. Use-case optimized data delivery repositories facilitate data self-service.
Impact and Result
- Understand your high-level business capabilities and interactions across them – your data repositories and flows should be just a digital reflection thereof.
- Divide your data world in logical verticals overlaid with various speed data progression lanes, i.e. build your data pipeline – and conquer it one segment at a time.
- Use the most appropriate database design pattern for a given phase/component in your data pipeline progression.
After each Info-Tech experience, we ask our members to quantify the real-time savings, monetary impact, and project improvements our research helped them achieve. See our top member experiences for this blueprint and what our clients have to say.
Average $ Saved
Average Days Saved
University of New Brunswick
Workshop: Build a Data Pipeline for Reporting and Analytics
Workshops offer an easy way to accelerate your project. If you are unable to do the project yourself, and a Guided Implementation isn't enough, we offer low-cost delivery of our project workshops. We take you through every phase of your project and ensure that you have a roadmap in place to complete your project successfully.
Module 1: Understand Data Progression
Identify major business capabilities, business processes running inside and across them, and datasets produced or used by these business processes and activities performed thereupon.
Key Benefits Achieved
Indicates the ownership of datasets and the high-level data flows across the organization.
Review & discuss typical pitfalls (and their causes) of major data management initiatives.
- Understanding typical pitfalls (and their causes) of major data management initiatives.
Discuss the main business capabilities of the organization and how they interact.
- Business capabilities map
Discuss the business processes running inside and across business capabilities and the datasets involved.
- Business processes map
Create the Enterprise Business Process Model (EBPM).
- Enterprise Business Process Model (EBPM)
Module 2: Identify Data Pipeline Components
Identify data pipeline vertical zones: data creation, accumulation, augmentation, and consumption, as well as horizontal lanes: fast, medium, and slow speed.
Key Benefits Achieved
Design the high-level data progression pipeline.
Review and discuss the concept of a data pipeline in general, as well as the vertical zones: data creation, accumulation, augmentation, and consumption.
- Understanding of a data pipeline design, including its zones.
Identify these zones in the enterprise business model.
- EBPM mapping to Data Pipeline Zones
Review and discuss multi-lane data progression.
- Understanding of multi-lane data progression
Identify different speed lanes in the enterprise business model.
- EBPM mapping to Multi-Speed Data Progression Lanes
Module 3: Develop the Roadmap
Select the right data design patterns for the data pipeline components, as well as an applicable data model industry standard (if available).
Key Benefits Achieved
Use of appropriate data design pattern for each zone with calibration on the data progression speed.
Review and discuss various data design patterns.
- Understanding of various data design patterns.
Discuss and select the data design pattern selection for data pipeline components.
- Data Design Patterns mapping to the data pipeline.
Discuss applicability of data model industry standards (if available).
- Selection of an applicable data model from available industry standards.