Improving Social and Economic Outcomes: The Value of Data Quality at the Federal Level

Enhancing federal performance with reliable and accurate data.

Analyst Perspective

Leveraging data quality to drive federal-level efficiency and effectiveness

Understanding the value of data quality at the federal level is essential. High-quality data is vital for informed decision making, effective policy making, and achieving the desired positive social and economic outcomes. Accurate and reliable data can help identify trends, address challenges, and measure the impact of government programs.

Data quality reduces errors and improves efficiency, since accurate and consistent data reduces the need for manual checks and corrections, freeing up valuable time and resources for other critical tasks. It also improves transparency and accountability. Collecting and sharing high-quality data provides greater transparency to the public, demonstrating to the taxpayer the government's commitment to achieving social and economic goals. The IT departments can help establish the necessary technology and infrastructure to collect and analyze data effectively while ensuring data security and privacy.

The value of data quality at the federal level cannot be overstated. History demonstrates that accurate and reliable data is crucial for making informed decisions, developing beneficial and sustainable policies, and achieving positive outcomes for society.

IT professionals must understand the importance of data quality and collaborate with other departments to establish and maintain the necessary policies, procedures, change controls, self-auditing techniques, and other key elements of a well-thought IT framework. Their successful implementation can lead to a more data-driven government that is better equipped to meet the needs of the public.

Paul Chernousov
Research Director, Industry
Info-Tech Research Group

Executive Summary

Info-Tech Insight

Fix data quality issues as close as possible to the source of data, while understanding that business use cases will each have different requirements and expectations from data quality.

Data is the foundation of your agency's knowledge

Data enables your agency to make decisions.

Reliable data is needed to facilitate data consumers at all levels of the organization.

Insights, knowledge, and information are needed to inform operational, tactical, and strategic decision-making processes. Data and information are needed to manage the business and empower key business processes such as citizen touchpoints, cross agency interactions and strategic planning.

Data should be at the foundation of your agency’s evolution. The transformational insights that Ministers and Agency leaders are constantly seeking can be uncovered with a data quality practice that makes high-quality, trustworthy information readily available to the business users who need it.

98% of companies use data to improve customer experience.
— Source: Experian Data Quality, 2019

Efficiently manage data quality across the data landscape

Good data quality is achieved by working efficiently.

Where possible, fix your data issues at source;
it’s typically cheaper and is less disruptive.

Focus your efforts on your most critical data,
the data that supports your core functions and services.

Not every agency requires the same standard of data quality;
fix for need, not perfection.

Work collaboratively;
good data quality requires collective working across business, data, and IT teams.

Source: Build Your Data Quality Program

Prevent the domino effect of poor data quality

Data is the foundation of good business decisions made at insight-driven agencies.

If there are problems with an agency’s underlying data, this can have a domino effect on many downstream business functions and their decision making.

Poor data quality in the implementation and migration to new systems can have the following effects:

• Project delivery times are prolonged when manual data quality fixes are employed.
• Costs increase for project staffing, infrastructure, and hardware when additional resources are required to complete the unforeseen work.
• Adoption and goal realization are negatively impacted with potential mistrust in the new system.

Three key challenges impacting the ability to deliver excellent customer experience:

95% of organizations indicated that poor data quality undermines business performance.

— Source: Experian Data Quality, 2019

Maintaining quality data will support more informed decisions and strategic insight

Improving your agency’s data quality will help realize the following benefits:

• Data-driven decision making: Business decisions should be made with a strong rationale. Data can provide insight into key agency questions, such as, “How can I provide better citizen satisfaction?”
• Citizen understanding: Improve citizen experience by using the right data from the system of record to analyze complete citizen views of transactions, sentiments, and interactions.
• Innovation leadership: Gain insights on your services, usage trends, and macro challenges to support decisions on innovations, new services, and service performance levels.
• Operational excellence: Make sure the right service is delivered rapidly and consistently to the right parties for the right cost structure. Automate processes by using the right data to drive process improvements.

Maintaining data quality is difficult

Avoid these pitfalls to get the true value out of your data.

• Data debt: Data debt hinders efficiency and can prevent you from achieving your desired process efficiencies.
• Lack of trust: If you don't trust your data, you won't use it effectively. Lack of confidence in data quality can negatively impact business outcomes.
• Liability: Poor quality data can cause your agency to fail to meet compliance standards, which may directly damage your reputation.
• Increased costs and inefficiency: Fixing bad data takes time, which reduces your agency's capacity for important initiatives and hampers your ability to make data-driven decisions.
• Barrier to adopting data-driven tech: Emerging technologies like predictive analytics and AI need accurate, complete, and current data to work well. You cannot be an effective data-driven agency with poor data.
• Bad citizen experiences: Using poor-quality data in your agency's services can impede your ability to provide effective service to your customers, leading to frustration and discouraging them from further engaging with your services.

— Source: “7 ways,” Experian Data Quality, 2020

Info-Tech Insight

Data quality suffers most at the point of entry. This is one of the causes of the domino effect of data quality – and can be one of the costliest forms of data quality errors due to error propagation. In other words, fix data ingestion, whether through improving your application and database design or improving your data ingestion policy, and you will fix a large majority of data quality issues.

All disciplines for data are primarily about building trust in your data

Trusted data, of known quality, drives improved business decision making and citizen customer experience for any Agency.

Accelerators for change

Data governance drives stewardship, supports the adherence of use to classification standards, and promotes the value of a business glossary for common understanding.

Modern data platforms can automate data lineage and access management and use AI to connect data based on shared attributes to promote use and consistency of data. They can also help monitor and manage the quality of data as it flows between systems and processes, so consumers know it is fit for their purpose.

Govern data as a key asset

Data quality will only be managed well if it is supported by a governance structure that sponsors, prioritizes investment, and enforces data quality practices.

Data quality means high-level, not perfection

Data from Info-Tech’s CIO Business Vision Diagnostic, which represents over 400 business stakeholders, shows that data quality is very important when satisfaction with data quality is low.

However, when data quality satisfaction hits a threshold, it becomes less important.

Info-Tech Insight

Data needs to be good, but truly spectacular data may go unnoticed.

Provide the right level of data quality, with the appropriate effort, for the correct usage. Info-Tech will help you to determine what “the right level of data quality” means, as well as create a plan to achieve that goal for your Agency.

Info-Tech’s methodology for data quality

Info-Tech Insight

“Data Quality is in the eyes of the beholder.” – Igor Ikonnikov, Research Director, Info-Tech

Key data quality elements

1. Define your agency’s data environment and business landscape

Data quality is a methodology and must be treated as such

A comprehensive data quality practice includes appropriate business requirements gathering, planning, governance, and oversight capabilities, as well as empowering technologies for properly trained staff, and ongoing development processes.

Here are some common examples of appropriate data management methodologies for data quality:

• The data quality team has the necessary competencies and resources to perform the outlined workload.
• There are processes that exist for continuously evaluating data quality performance capabilities.
• Improvement strategies are designed to increase data quality performance capabilities.
• Policies and procedures that govern data quality are well-documented, communicated, followed, and updated.
• Change controls exist for revising policies and procedures, including communication of updates and changes.
• Self-auditing techniques are used to ensure business-IT alignment when designing or recalibrating strategies.

Data quality can be defined by four key quality indicators

The quality of your data can be measured using a simple indicator test, like measuring the acidity of a substance with a litmus test. As you learn about common root causes of data quality problems in the following slides, think about these four quality indicators to assess the quality of your data:

• Completeness – Closeness to the correct value. Encompasses accuracy, consistency, and comparability to other databases.
• Usability – The degree to which data meets current user needs. To measure this, you must determine if the user is satisfied with the data they are using to complete their business functions.
• Timeliness – Length of time between creation and availability of data.
• Accessibility – How easily a user can access and understand the data (including data definitions and context). Interpretability can also be used to describe this indicator.

— Adapted from: IGI Global

Info-Tech Insight

Quality is a relative term. Data quality is measured in terms of standards. Perfect data quality is both impossible and a waste of time and effort.

Data quality diagnostic value

Prioritize business use cases with data quality dimensions.

Complete this diagnostic for each major business use case. The output from the Data Culture Diagnostic and the Business Capability Map should help you understand which use cases to address.

Involve all key stakeholders involved in the business use case. There may be multiple agencies involved in a single use case.

Prioritize the business use cases that need the most attention pertaining to data quality by comparing the scores of the Importance and Confidence dimensions.

If there are data elements that are considered of high importance and low confidence, then they must be prioritized.

Sample Scorecard

Poor data quality develops due to multiple root causes

Understand the underlying causes of why those indicators can point to poor data quality.

Monitor the usability, completeness, timeliness, and accessibility of the agency’s data. If it is suffering, one or more of these root causes are likely plaguing your data:

These root causes of poor data quality are difficult to avoid, not only because they are often generated at an agency’s beginning stages, but also because change can be difficult. This means that the root causes are often propagated through stale or outdated business processes.

2. Analyze your priorities for data quality fixes

Business context and data quality

Establish the business context of data quality improvement projects at the agency level to find common goals.

Start your data quality project evaluation by understanding the business context. This will ensure the data improvement strategy is business driven. You will then determine which agencies use data and create a roadmap for prioritizing agencies for data quality repairs.

Your business context is represented by your Agency’s vision, mission, goals and objectives, differentiators, and drivers. Collectively, they provide essential information on what is important to your agency, and some hints on how to achieve that.

Leverage the business view to establish the data view. In this step, you will gather important information about your business view and interpret the business view to establish a data view.

Not every agency uses data to the same extent

Understanding where data lives can be challenging as it is often in motion and rarely resides in one place. There are multiple benefits that come from taking the time to create a data flow diagram.

Map out the flow of data to help provide clarity on where the data lives and how it moves through the agency’s systems.

A visual of where and when data moves helps you understand who is using data and how it is being manipulated at different points.

A data flow diagram will allow you to elicit how data is used in a different use case.

A data flow diagram can provide value by allowing an organization to adopt a proactive approach to data quality. Save time by knowing where the entry points are and where to look for data flaws.

Not every agency requires the same standard of data quality

To prioritize your agencies for data quality improvement projects, you must analyze the relative importance of the data they use to the business. The more important the data is to the business, the higher the priority is to fix that data.

Business value and business impact determine the importance of data.

Ensure that the project starts on the right foot by articulating your Data Quality Problem Statements

Before you can identify a solution, you must identify the problem with the agency’s data.

Identify the symptoms of poor data quality and articulate the problem.

Identify and describe the problems that the agency sees in its data quality through a step-by-step approach.

Identify the symptoms of the problem. The following Ws will help you describe the symptoms of the data quality issues:

• What: Define the symptoms and feelings produced by poor data quality in the agency.
• Where: Define the location of the data that are causing data quality issues.
• When: Define how severe the data quality issues are in frequency and duration.
• Who: Define who is affected by the data quality problems and who works with the data.

Info-Tech Best Practice

Symptoms vs. problems. People often identify a list of symptoms of a problem and mistake those for the problem. Identifying the symptoms helps to define the problem, but symptoms do not help to identify the solution. The problem statement helps you to create solutions.

Create your strategy for improving data quality

Assess IT’s capabilities and competencies for data quality and plan to build these as the agency’s data quality practice develops

Before you can fix data quality, make sure you have the necessary skills and abilities to fix it correctly.

The following IT capabilities are developed on an ongoing basis and are necessary for standardizing and structuring a data quality practice:

• Meeting business needs
• Services and projects
• Policies, procedures, and standards
• Roles and organizational structure
• Oversight and communication
• Data quality of different data types

Data handling and remediation competencies

• Data standardization: Formatting values into consistent standards based on industry standards and business rules.
• Data cleansing: Modification of values to meet domain restrictions, integrity constraints, or other business rules for sufficient data quality for the organization.
• Data matching: Identification, linking, and merging related entries in or across sets of data.
• Data validation: Checking for correctness of the data.

Improvement actions should be developed after these capabilities and competencies are assessed for a current and desired target state, and followed in order to build your data quality practice.

A roadmap must be generated after target dates are set to create your data quality practice development strategy.

Receive sign-off from IT regarding feasibility

Before engaging IT in data quality projects to fix the agencies’ data, IT must assess feasibility of the data quality improvement plan. A feasibility analysis is typically used to review the strengths and weaknesses of the projects, as well as the availability of required skills and technologies needed to complete them. Use the following workflow to guide you in performing a feasibility analysis:

Info-Tech Best Practice

While the PMO identifies and coordinates projects, IT must determine how long and for how much.

Evolution of data quality management

As you improve your data quality practice and move from reactive to stable, don’t rest and assume that you can let data quality keep going by itself. Rapidly changing citizen requirements or other pains will catch up to your agency and you risk falling behind again. By moving to the proactive and predictive end of the maturity scale, you can stay ahead of the curve. By following the methodology, the data quality practices at your agency will improve over time, leading to the following results:

Source: Global Data Excellence, Data Excellence Maturity Model

3. Establish your agency data quality program

Create a data lineage diagram to map the data journey and identify the areas to fix

It is important to understand the various data that exist in the agency, as well as which data are essential to business function and require the highest degree of quality efforts.

Visualize your databases and the flow of data. A data lineage diagram can help you and the data quality improvement team visualize where data issues lie. Keeping the five-tier architecture in mind, build your data lineage diagram.

Reminder: Five-Tier Architecture

Identify metrics at the agency level to track data quality improvements

As you improve the data quality for specific agencies, measuring the benefits of data quality improvements will help you demonstrate the value of the projects to the agency.

Use the following table to guide you in creating business-aligned metrics:

Relate data quality metrics to the four main value drivers. Each agency will contribute to each of these drivers to a different degree:

1. Citizen understanding
2. Operational excellence
3. Risk management
4. Innovation leadership

Info-Tech Insight

Relating data governance success metrics to overall business benefits keeps agency management sponsors engaged because they are seeing actionable results. Review metrics on an ongoing basis with those data owners/stewards who are accountable, the data governance steering committee, and the executive sponsors.

Begin your data quality improvement project starting with the highest priority agency

Identify the highest priority agency once you have a prioritized list for your data quality improvement projects. Once you have initiated and identified solutions for the first agency, tackle data quality for the next agency in the prioritized list.

Now that you have a defined problem, revisit the root causes of poor data quality

You previously fleshed out the problem with data quality present in the agency chosen as highest priority. Now it is time to figure out what is causing those problems.

In the table below, you will find some of the common categories of causes of data quality issues, as well as some specific root causes.

4. Grow and sustain your data quality program

Develop data quality improvement strategy for each root cause

Example: Fix data quality issues by improving system/application design.

Technology

Application Interface Design

• Restrict field length – Capture only the characters you need for your application.
• Leverage data masks – Use data masks in standardized fields like zip code and phone number.
• Restrict the use of open text fields and use reference tables – Only present open text fields when there is a need. Use reference tables to limit data values.
• Provide options – Use radio buttons, drop-down lists, and multi-select instead of using open text fields.

Data Validation at the Application Level

• Validate data before committing – Use simple validation to ensure the data entered is not random numbers and letters.
• Track history – Keep track of who entered what fields.
• Cannot submit twice – Only design for one-time submission.

People and process

Training

• Data-entry training – Training that is related to data entry, creating, or updating data records.
• Data resolution training – Training data stewards or other dedicated data personnel on how to resolve data records that are not entered properly.

Continuous Improvement

• Standards – Develop application design principles and standards.
• Field testing – Field data entry with a few people to look for abnormalities and discrepancies.
• Detection and resolution – Abnormal data records should be isolated and resolved ASAP.

Application Testing

• Thorough testing – Application design is your first line of defense against poor data. Test to ensure bad data is kept out of the systems.

Sustaining your data quality requires oversight through a data governance

Quality data is the ultimate outcome of data governance and data quality management.

Data governance enables data quality by providing the necessary oversight and controls for business processes in order to maintain data quality.

There are three primary groups (below) that are involved in a mature governance practice. Data quality should be tightly integrated with all of them.

Integrate the data quality management strategy with existing data governance

Ongoing and regular data quality management is the responsibility of the data governance bodies of the organization.

The oversight of ongoing data quality activities rests on the shoulders of the data governance committees that exist in the organization.

There is no one-size-fits-all data governance structure. However, most organizations follow a similar pattern when establishing committees, councils, and cross-functional groups. They strive to identify roles and responsibilities at a strategic, tactical, and operational level:

Develop a business data quality dashboard to show improvements or a sudden dip in data quality

One tool that the data steward can take advantage of is the data quality dashboard. Initiatives that are implemented to address data quality must have metrics defined by business objectives in order to demonstrate the value of the data quality improvement projects. In addition, the data steward should have tools for tracking data quality in the agency to report issues to the data owner and data governance steering committee.

Approach to creating a business-facing data quality dashboard:

1. Schedule a meeting with the functional unit to discuss what key data quality metrics are essential to their business operations. Consider the business context, functional area, and subject area analyses you completed in Phase 1 as a starting point.
2. Discuss how to gather data for the key metrics and their associated calculations.
3. Discuss and decide on the reporting intervals.
4. Discuss and decide on the unit of measurement.
5. Generate a dashboard similar to the example. Consider using a BI or analytics tool to develop the dashboard.

Data quality management must be sustained for ongoing improvements to the agency’s data

Data quality is never truly complete; it is a set of ongoing processes and disciplines that requires a permanent plan for monitoring practices, reviewing processes, and maintaining consistent data standards.

Setting the expectation to stakeholders that a long-term commitment is required to maintain quality data within the organization is critical to the success of the program.

A data quality maintenance program will continually revise and fine-tune ongoing practices, processes, and procedures employed for organizational data management.

Data quality management is a long-term commitment that shifts how an organization views, manages, and utilizes its corporate data assets. Long-term buy-in from all involved is critical.

“Data quality is a process. We are trying to constantly improve the quality over time. It is not a one-time fix.”

– Akin Akinwumi, Manager of Data Governance, Startech.com, in Build Your Data Quality Program, Info-Tech Research Group

Related Info-Tech Research

Bibliography

Amidon, Kirk. "Case Study: How Data Quality Has Evolved at MathWorks." The Fifth MIT Information Quality Industry Symposium. 13 July 2011. Web. 19 Aug. 2015.

Boulton, Clint. “Disconnect between CIOs and LOB managers weakens data quality.” CIO. 05 February 2016. Accessed June 2020.

COBIT 5: Enabling Information. Rolling Meadows, IL: ISACA, 2013. Web.

Cohen, Ira. “The End to a Never-Ending Story? Improve Data Quality with AI Analytics.” anodot. 2020.

“DAMA Guide to the Data Management Body of Knowledge (DAMA-DMBOK Guide).” First Edition. DAMA International. 2009. Digital. April 2014.

"Data Profiling: Underpinning Data Quality Management." Pitney Bowes. Pitney Bowes - Group 1 Software, 2007. Web. 18 Aug. 2015.

Data.com. “Data.com Clean.” Salesforce. 2016. Web. 18 Aug. 2015.

“Dawn of the CDO." Experian Data Quality. 2015. Web. 18 Aug. 2015.

Demirkan, Haluk, and Bulent Dal. "Why Do So Many Analytics Projects Fail?" The Data Economy: Why Do so Many Analytics Projects Fail? Analytics Magazine. July-Aug. 2014. Web.

Dignan, Larry. “CIOs juggling digital transformation pace, bad data, cloud lock-in and business alignment.” ZDNet. 11 March 2020. Accessed July.

Dumbleton, Janani, and Derek Munro. "Global Data Quality Research - Discussion Paper 2015." Experian Data Quality. 2015. Web. 18 Aug. 2015.

Eckerson, Wayne W. "Data Quality and the Bottom Line - Achieving Business Success through a Commitment to High Quality Data." The Data Warehouse Institute. 2002. Web. 18 Aug. 2015.

“Infographic: Data Quality in BI the Costs and Benefits.” HaloBI. 2015 Web.

Lee, Y.W. and Strong, D.M. “Knowing-Why About Data Processes and Data Quality.” Journal of Management Information Systems. 2004.

“Making Data Quality a Way of Life.” Cognizant. 2014. Web. 18 Aug. 2015.

"Merck Serono Achieves Single Source of Truth with Comprehensive RIM Solutions." www.productlifegroup.com. ProductLife Group. 15 Apr. 2015. Web. 23 Nov. 2015.

Myers, Dan. “List of Conformed Dimensions of Data Quality.” Conformed Dimensions of Data Quality (CDDQ). 2019. Web.

Redman, Thomas C. “Make the Case for Better Data Quality.” Harvard Business Review. 24 Aug. 2012. Web. 19 Aug. 2015.

RingLead Data Management Solutions. “10 Stats About Data Quality I Bet You Didn’t Know.” RingLead. Accessed 7 July 2020.

Schwartzrock, Todd. "Chrysler's Data Quality Management Case Study." Online video clip. YouTube. 21 April. 2011. Web. 18 Aug. 2015

“Taking control in the digital age.” Experian Data Quality. Jan 2019. Web.

“The data-driven organization, a transformation in progress.” Experian Data Quality. 2020. Web.

"The Data Quality Benchmark Report." Experian Data Quality. Jan. 2015. Web. 18 Aug. 2015.

“The state of data quality.” Experian Data Quality. Sept. 2013. Web. 17 Aug. 2015.

Vincent, Lanny. “Differentiating Competence, Capability and Capacity.” Innovation Management Services. Web. June 2008.

“7 ways poor data quality is costing your business.” Experian Data Quality. July 2020. Web.

“What Is Information Quality.” IGI Global.

Wang, John, et al. “Data Mining in Public Administration.” IGI Global, 1 Jan. 2008.

Danielson, David R. “Usability Data Quality.” IGI Global, 1 Jan. 2006.

“Glossary.” Global Data Excellence, 9 Nov. 2020.