Data Quality Validation Checks
A data quality validation check is used to analyse a data set.
You can create different types of validation checks for specific columns to determine whether a record has values that pass or fail according to your check. For example, if you create a data quality rule where a column must be between 1 and 9, if a value of 10 is in this column, this row will be listed as fail.
A data quality rule is created similarly to an alert, except you will additionally have to add validation checks. You will specify a column for each validation check.
A data quality validation check will provide a status of ‘Pass’ or ‘Fail’ for each row. It can be viewed as a column in your results table once you have run this rule. You can also use these columns as slicers in your results to find which rows have specifically passed and failed when you have multiple column validation checks.
Validation Check Types
| Type | Description | Example |
|---|---|---|
| Database Function | A Database Function can be used to execute a boolean SQL expression. | For example, it can be used to check the data type of values in your selected column, such as whether a column with a string data type contains numeric values. |
| Date Validation | Date Validation will check that a column contains dates within a specific range. | You can check that your selected date falls within your defined date range. |
| Fixed Value | Fixed Value column validation will check that a column contains the values that you have provided. | You can provide the values 1, 10, and 50 and check that a int or varchar column contains these numbers. |
| Format | A format validation column will check that your selected column conforms to a specific format. | You can check that a varchar column contains emails. |
| Number Validation | Number Validation will check that a column contains number values within a specific range. You can check that your selected value falls within your defined specific number range. | You can create a check of values less than 750, which will determine whether you have any products that have fallen below the safety stock level of 750. |
| Reference Lookup | A Reference Lookup will check that your column values exist within a reference column. | You can use a reference lookup to compare two columns, one column in this data quality rule and the other column would be from a different rule. You can compare the ID column of both rules to check that there are no missing records when you cross-reference your data. |
How to Add a Data Validation Check
To add a data validation check, click on Add Column Validation.
This will expand a slideout where you can choose one of the validation check types described above.
You will need to name this validation check so that you can easily identify it later.
You will also choose the column that you would like to check.
If you would like to learn more about a validation type while using Loome, the Read More link will direct you to the relevant documentation page.
Please follow the guides in the next section to learn how to create each validation type.