Quick Start Guide
Getting Started
You will need to have a few things ready before you can start creating alerts, references or glossaries. You will only have to set up these steps once and you can then use them over and over again for your three rule types.
| Step | Description | Related Documentation |
|---|---|---|
| Step 1: Setup an Agent | This will be the worker that orchestrates your data and completes the actions that you have ruled out. | Here is a link to learn more about the agent |
| Step 2: Add Connections | These are your data sources and targets. | Read more about connections |
| Step 3: Create a Project | This will categorise your rules (alerts, references and glossaries). | Learn about projects here |
| Step 4: Add users to Application Roles | These roles will give permissions to your selected users. You can select which users can create rules and which can see read-only results. | Find more information about user application roles here |
| Step 5: Create your first Rule | A rule can be an alert, reference or glossary. A rule will trigger notifications and instigate a business workflow. | Read about our rule types here |
The following sections will show you how to complete each step needed to create and run a rule.
See our application roles page to learn which roles have permission to take the following actions.
How to Add an Agent
Follow the instructions below to add an agent. You will have to download and run the Loome agent installer for your platform of Windows, Linux, Docker Run or Azure Container instances.
Click on Agents in the top-right drop down.
Click on Setup Agent.
This will expand the slide-out where you can select your platform. Choose from Windows, Linux, Docker Run or Azure Container instances.
Once you have selected your platform, you will be shown the steps you can follow to install an agent. This varies for the platform that you have selected. Please read more about the agent, its requirements and detailed instructions here.
Learn how to install the agent for Windows and Linux here.
Learn how to install the agent for Docker and Azure Container instances here.
How to Add a Connection
Click on Connections in the top-right settings dropdown menu.
Click on Add Connection to start creating a connection to your data platform.
Select a connection type.
Once you have selected a connection type, follow the next steps to complete the fields you need to create a connection.
First, provide a friendly name to easily identify this connection.
If required, you can then add your username and password (or your Personal Access Token) in these fields to provide your username and password separately from the connection string. What details you provide here can vary depending on the connector type. This would be the same credentials you use to log into the account of this data source or target. For example, if you were adding a SQL Server connection, this would be your SQL Server account username and password.
Then add your connection string.
Find connection string templates for Microsoft SQL Server, Snowflake Data Warehouse and Azure Synapse SQL in each of these links.
Select the agent that will validate this connection.
Next, choose whether this connection will be available in all projects or selected projects.
If it should only be available in your chosen projects, click ‘Selected Projects’ and tick the checkbox beside each project you would like to use this connection. It will not be available in other projects.
Once you have completed these fields, save at the top right corner.
You will now be able to find this connection when creating and editing rules.
Read more about connections here.
How to Add a Project
You can create a new project from the left-hand sidebar.
Provide a friendly name to easily identify the type of rules that you will add to this project.
You can then add members to your project. Learn more about project roles here.
Click on Create Project in the bottom right corner and you will then be able to select this project on the homepage or left-hand project slide-out.
Learn how to manage projects here. And learn more about project roles here.
How to Add a User to an Application Role
Find the application role page from the top-right settings menu.
Choose the role that you would like to add a user to from the tabs at the top of the page and click on Add Members.
Choose to add a user or an API Key.
If you want to add a user, type in the email address of the user and then click +. (You can add multiple users at once using a semi-colon.)
Click on Submit to add this user.
If you want to add an API Key, choose the API Key from the dropdown, and click on Submit.
If they have only been sent an invitation their status will remain as ‘User Invitation’ until they log in, in which their status will change to ‘User’.
Please read more about application roles and learn which roles provide which permissions here.
How to Create a Rule
A rule will trigger notifications and instigate a business workflow.
There are five types of rules.
Alert: An alert is used to define a data quality or process compliance exception that you want to keep track of and assign to someone to resolve.
Glossary: A glossary will annotate an existing list of terms, fields or tables with explanatory notes and classifications.
Reference: You can use a reference to define a data set that requires a data steward to manually update additional attributes for reporting purposes.
Data Quality Rule: Define a set of validation checks to be run for specific columns in your data set.
Reconciliation: Check for differences between two data sets. You can compare two data sets in one rule, and view both in the one results table to see highlighted mismatches and orphan records from each data set.
In the project you created earlier, click on Create New Item. For this example, we will be creating an alert.
If you have chosen to create a glossary or reference, you will also be asked to choose whether you would like to source your data from a source or manually enter your data. Continue following this guide if you are creating an alert or a rule that uses source data, but if you would like to learn more about manually adding records to your data set follow the guide in this link.
The first step in creating a rule, no matter the type, is providing a name for the rule. Rule names should be user friendly, unique, and summarizes the purpose of the rule.
You also have the option to add a description for this rule. Here you can describe the rule and its purpose in more detail.
Select an agent that will execute this rule. An agent contains all the code that allows rules to run.
Choose to add Manually Populated Data
If you have chosen a glossary or reference, you can also choose to add manually created records to your data set.
If you have chosen another rule type, you can move onto the next step of this guide.
You can choose ‘Manual Data’ or ‘Both Source Query And Manual Data’ as your data source.
If you choose these options, you can choose to add additional records to your data set on the results page, or you can create a data set comprised entirely of columns and rows that you have added in Loome.Learn more about creating a manual rule using this guide here.
Choose your Connections
Rules consist of two connections; the source connection and the output connection. First select the source connection, your rule’s query will be retrieving data from this connection. Then select the output connection and a dedicated schema will be created in this database connection.
These can be the connections we created earlier or you can add a new connection using the ‘Add’ button beside the connection dropdown fields.
A reconciliation rule will ask you for a source connection and a comparison connection. The two datasets will be compared in the results table. Learn more here.
Write your Query
Provide a query that will retrieve data from your source connection for this rule.
This query should be written using the same SQL variant that your source connection utilizes. Loome Monitor will validate the query as you type and provide any errors or warnings below the query box.
A reconciliation rule will ask you for two queries. One for each source connection. The two datasets will be compared in the results table. Learn more here.
Choose your Key Columns
In this next window, select one or more Key Columns for the rule. Examples of key columns are unique identifiers such as primary keys or a combination of columns to form a composite key. In the event of Key Columns not being entirely unique Loome Monitor will not behave as expected.
You will need to choose at least one key column and leave at least one column unselected.
Please note that rules that use a SQL connection as a target connection will use clustered indexes. If you choose any individual key column size that exceeds 900 bytes or if the sum of all key column sizes exceed 2600 bytes, you will need to confirm your key column selection as these columns can cause your data to be truncated.
Select an Existing Column to Assign Users to Records (Optional)
In this next window you can choose to populate the ‘Assigned To’ field from your own data, rather than manually completing this field after your results have been created. This is optional and you can choose to skip this step.
All columns in your data source will be available to choose from.
In order to populate the Assigned to fields with users, this column must contain values in an email format.
Choose an existing column from your data source that contains the email addresses you would like to assign to each record and click Next.
Or you may leave it blank and continue Next without choosing a column. You can later assign users in the results table.
If you have already added a user to an assigned to field, this user will not be replaced with the value in your selected column. The values in your selected column will only populate empty fields.
Create Custom Fields
You can choose to create custom fields to capture feedback for your results in this next step, but it is not a requirement and you can proceed without adding any custom fields. Loome will create a dedicated schema for your custom field metadata and it will be independent from the results table generated from this rule.
To create a custom field, click on Add Custom Field and a slide-out will be expanded with the option to choose from six different field types.
Read more about custom fields here.
Custom Field Lockout Column (Optional)
In rules that use source data, you can optionally choose a boolean or bit type column from your source data query to indicate whether a record’s custom fields can be edited. It will lock rows with a true value within your rule and prevent the editing of their custom fields.
When a row has a true value in this column, the custom fields of that row cannot be edited, and if false, the row can be edited. (You can view this in the results table after you have run the rule.)
You may skip this step and leave it unselected if you do not wish to lockout any custom fields.
Data Quality Rule Step
If you are not creating a Data Quality rule you can proceed to the next step.
If you are creating a Data Quality rule you will be asked to add column validation. Learn more here.
Complete the Rule
Click on Create to save and complete this rule.
Run the Rule
You will be shown a page where you have the option to either return to your project, edit, or run this rule.
Run this rule to view the results it will generate.
Once you click on Run, you will be able to see the progress of this rule as each step is listed under ‘Execution Steps’.
Once this rule has run successfully, you will have your results ready so you can view and edit your results as explained below or you can add communication steps.
Add Communication Rules
Once you have run a rule, you can choose to add Communication Rules. Click on Edit on the review page shown above or find the edit button from the project page. You can then find the communication rules tab on the left of the page.
You can choose to create a branch that will send communications only on specific conditions or send an overall email to a target audience.
Learn more about communication rules on this page here.
View and Edit your Results
Click on the Results button, which is the table icon in the top right of these execution steps, to be directed to your Results page.
View and filter your results using the slicers above each column beside each column name.
Edit your results using the edit button on the left of the row you want to edit.
This will display the record page where you can edit who is assigned to this result, whether a result is ignored, and each custom field you added when creating or editing this rule.
You can add values to each field.
Once you save this row, you will find your changes in the results table.
