Why Does Data Validation Matter?

“If you can’t measure it you can’t improve it” – Peter Drucker.

There’s a big difference between the price of a tool, and the full-scope picture of what it costs.  As a result, we typically answer questions about cost  with “It depends”.

  • It depends on how much you currently rely on humans to perform manual date entry.
  • It depends on how many tries it currently takes you to get your measurement data as accurate as you need it to be.
  • And it really depends on whether you fear any repercussions as a result of inaccurate reporting.

If we look at a very simplified picture of how data typically flows from the field to its final report, we can see how most companies rely on manual efforts to validate their data.


The red in both flows is where there is human intervention required and in the bottom flow chart there are many more human intervention steps required.

What we know is that humans are not good at comprehensively reviewing large gobs of data and trying to find the erroneous data can be an exercise in extreme frustration.

On the flip side, computers are fantastic at doing this type of thing, yet most companies, even within the  age of “Big Data” and “Analytics”, still have not automated their data validation. So what’s the holdup?

The cost of entry into something new is what scares most people — so let’s break it down.

For an easy to follow case study, I’ve chosen Natural Gas as an industry example because it is the most obvious and easiest to improve.

Currently, Natural Gas companies utilize “Flow Computers” enforce a data standard based on API 21.1, but automated data validation is very applicable to any measurement. (Liquids, Gas, Solids, Multiphase) and for that matter any large amounts of data that need to be evaluated within a set of rules.

Inaccuracies and manually manipulating data can cost companies a lot of money, but when you manually manipulate data it is like boiling a frog; You don’t know it’s costing you money because they are costs that get buried in salaries spread across countless employees, and is money that has been wasted like this for decades – so what’s the big deal?

The big deal is that you are, whether you realize it or not, positioning yourself for “death by inefficiency”.

Everyone knows that doing things a certain way, just because you always have, is a fool’s position. The only way to truly determine your best course of action is to evaluate the cost of doing things the same way as always against the short and long-term costs of change.

Once you truly understand the cost and shortcomings of your inefficient manual process, all we need to do is justify the cost of changing over to an automated system. Here’s how we do it:

  1. Estimate how much of your employee’s time goes into generating a report and then double or even triple it to account for revisions
  2. Get an estimate from a company who understands data flow, business needs and SCADA systems. (We would like you to contact BettsM for this 😊)
  3. Perform a fair comparison of real-world costs

Starting to see the advantages to an automated system – don’t settle for just any solution!

BettsM Controls sells Quantum EFM; a data validation and editing tool that also provides reporting trending, offline data entry, field data capture, telecom analysis and more.

It’s designed to act as your single source of truth, providing an audit trail (for those pesky court cases that shippers and receivers tend to get engaged in) and  provides all your government reporting requirements, equipping your company VP to tell the Government, “Yes, we are in regulatory compliance” without losing a single night’s sleep.

If we revisit our original quote “If you can’t measure it you can’t improve it” I suspect that most people reading this article have no idea how much it costs to get the “measured data” for the company and probably even less awareness of how much human fudging may have gone into those production numbers.

Yet, we understand a solution like this isn’t for everyone.

If you are the type of person that drives your car by looking at everyone else to maintain your speed and trajectory rather than drive it yourself. Then by all means don’t do anything about this opportunity.

However, if you use your speedometer and prefer the open road to staring at the bumper ahead of you – then this is your call to action.

Get an estimate and evaluation done today at www.bettsm.com.