Phone: (972) 717-5690
1701 W. Northwest Hwy, Suite 100 Grapevine, TX 76051
Irving, TX 75014
What does your data REALLY look like?
System data values often differ significantly from what is expected
Years of data migration experience has taught UCT experts that real data often contains unexpected values. This same data is often locked inside a "data base" with a proprietary or legacy format or no standardized query interface. Just seeing what data you have can be a challenge. Cleaning it up may be all but impossible. The UCT toolset includes data analysis and cleanup features that can show you your data and convert it to acceptable values.
We apply our data migration experts and process to data analysis
UCT applies expertise, our Agile methodology, and our toolset to meet a data analysis and cleanup challenge. Our experts use DCA to build the "audit" maps that will report the data values that exist in the system. DCA automatically populates the audit maps with the business logic that is needed to report data values. These maps can be used as-is or customized to add or remove logic as required. When the maps are ready, DCA generates data analysis code directly from them. When the code is executed, messages are created that report the values in the system. DCA Reporting gathers the messages into reports that summarize the findings and identify example records that contain the values.
Various kinds of data anomalies can be identified
There are many different kinds of data anomalies. UCT's methods can bring all kinds of data anomalies to light through the simple logic in the audit map.
Common kinds of data anomalies exposed by audit maps
- Unexpected values - Older technologies just didn't have features to enforce data integrity like today's technologies do. Data was hand entered and the user was expected to manually apply standards to the entered values. A simple field like GENDER that is documented as having only 2 valid values (M and F) might in reality have many values (M, F, m, f, U, u, blank, zero, and so on).
- Deleted values - Sometimes data is not completely deleted. Sometimes a whole record is composed of many pieces of separately stored data. When a parent record is deleted it may leave various disconnected pieces undeleted in other data files.
- Invalid or unexpected data relationships - Can a person die before they were born? In this simple example, the obvious answer is "No". Unfortunately, the data in a system might disagree.
- Data of an invalid data type - Sometimes you find that alphabetic data is stored in numeric fields or vice versa. This is a special kind of invalid value because it can have a disastrous impact on an unsuspecting program.
- Data with invalid characteristics - The documentation for a field may say that it is always upper case, it contains no spaces, its maximum length is 2 characters, it is never empty, it is always between 1 and 100, or it has any number of other characteristics like these. The real data may not conform to these rules.
When bad data is detected, it can be fixed
Once bad data is detected, additional map logic can fix it. Maps that were originally created to report data anomalies can be reused as the foundation for the maps that will fix the data. Business experts simply replace the reporting logic with data conversion logic that changes the data to a valid value. When a UCT customer is migrating to a different system, data is cleaned up as part of the data migration effort such that the clean data is loaded to the new system. Other times, data is cleaned up and applied to the original system as a data migration pre-step. The UCT solution can be used regardless of whether the cleaned data will be applied to the original system or to a new one.
UCT can help
If you need to see what's inside your system or clean up the bad data that you know is in it, UCT has the expertise, methodology, and tools to simplify the task. For more information about UCT's data analysis and cleanup services, contact Rae Albertini at (972) 717-5690 or submit our online information request form.