Three data validations you can do right now to make your users happier.

Three data validations you can do right now to make your users happier.

For all of you who edit GIS data with ArcFM the most frustrating problem you run into is getting ArcFM QA/QC errors. What is worse is editing a set of data and getting QA/QC errors on attributes that you did not even edit. ArcFM QA/QC was designed to prevent errors from occurring during an edit transaction, not for finding existing errors in your database. Existing errors need to have a specific strategy to detect and then plan a corrective action. Below are a few simple steps you can take using GeoData Sentry from Laurel Hill GIS, to detect systematic errors

Find the invalid values.

ArcGIS and ArcFM do a great job of maintaining data integrity for domain controlled columns by only allowing the values in a domain to be entered through a dropdown. Since invalid values are not allowed for new features, they cannot be entered. If invalid values exist and they just happen to be part of the edit transaction, the ArcFM QA/QC Error will be thrown. This is how it is designed, but it may not be optimal for getting a clear picture of the scope of the problem. Finding and reacting to an occasional error in the database is normal, but constantly having to make data corrections while performing database maintenance does not instill confidence in the geodatabase.

The solution for this problem is to generate domain tests with GeoData Sentry. GeoData Sentry will generate one domain test for each column with a domain assigned, additionally, for each subtype with a domain assigned. These tests can be run Ad-Hoc or on a schedule to validate the entire geodatabase. By assessing the entire geodatabase, patterns of error can be identified. It may be that all domain errors in the geodatabase can be traced back to an original conversion effort, or possibly a data model change made mid project where the data was not aggregated correctly to match the new domain values or even a wayward Autoupdater is incorrectly updating values. Regardless of the cause, proactively testing the database will yield a type and magnitude of error that can be prioritized and corrected accordingly.

Make sure your ArcFM required fields are populated.

The problem is the same as finding invalid values. Occasionally finding unpopulated fields that are required may not be a hardship, but over time, editors and data administrators must have confidence that the data and the ArcFM Configuration are correct.

The solution for this problem is to create a NULL test suite with GeoData Sentry. Each required field if the database should have a NULL test to identify unpopulated values. This is done by simply choosing the columns that are required and creating the tests. These tests can be run Ad-Hoc or on a schedule to validate the entire geodatabase. Once again, by testing the entire database a baseline of error can be determined, and a proactive corrective action can be applied. These values are required, so correction of this type of error should be a high priority. A byproduct of testing all required fields is that you may determine that there are errors with the ArcFM configuration. The issues could simply be that a column may be set to required that should not be or visa-versa.

Make sure the attributes used in your stored displays for symbology are populated and valid.

If you have planned ahead, your stored displays have an obvious symbol such as a large red question mark assigned when the values used for symbology are NULL or invalid. If your stored display is not configured this way, your features may not display. This could have severe consequences for editing and other map based uses of the data. As noted above there are several potential ways these values can become invalid. Once again, you may be editing features in an ArcFM session and get QA/QC errors for attributes that you are not even editing.

The solution for this problem is to create a NULL test suite with GeoData Sentry. As with the required fields these errors should be given a high priority for correction since they will affect the user’s view of the data.

These three validations will help identify the most common types of errors within your geodatabase.

If you have questions about the quality of your geodatabase feel free to send us an email at dataquality@laurelhillgis.com.

No comments (Add your own)

Add a New Comment


code
 

Comment Guidelines: No HTML is allowed. Off-topic or inappropriate comments will be edited or deleted. Thanks.