26th of September, 2023

Service

Organization Management

We have now implemented a new sign up and management experience for your Loome Organization. As an owner of a Loome Organization, you can manage your Loome tenants, user and API access, provide billing and technical contacts and view subscription details.

New users will also be redirected to this application when signing up to Loome, and can sign up for Loome Assist, Integrate, and Monitor in one location.

We will be providing further functionality via this app in future updates.

API Keys

API Keys have been enhanced in this release and standardized across Loome Assist, Integrate, and Monitor.

All keys now employ an expiry date. As part of the migration this will be set at 2 years for existing keys, but is configurable by you and will apply the next time a token is acquired for an API key.

You can also configure firewall rules to lock down what IP addresses an API Key can be used from.

API Keys now require you to assign permissions before they can be used. API Keys can be assigned an administrator role at the tenant level, or you can choose to assign an API key to a specific role in a specific project in order to limit its access. This is done via the existing user controls at the tenant and project level for user access. You will now be presented with the option to add users or add API Keys when assigning permissions.

As part of the migration, all existing API Keys will be added to the administrator role at the tenant level, and can then be modified by you as needed.

Additionally, new API Keys now require a separate client secret to operate, this adds a level of security when communicating API Keys to users. This will not affect existing API Keys, but you may chose to retire these existing API Keys and create new, more secure API Keys as needed.

API Keys can continue to be managed by tenant admins from within Loome Integrate, Monitor and Assist, or by organization admins via the new organization management app.

Schedules

We have introduced a new scheduling system that will remove issues we have had with the current system around daylight savings time changes, and various issues calculating next executions. We have moved to using a system based around CRONs, with a new wizard to create schedules. You can choose to use the wizard to build your schedule, or you can provide one or more CRONs to directly control your schedule.

We will migrate all existing enabled schedules to the new system automatically at the time of the release. While we take all precautions to ensure there will be no interruptions, we recommend that you check your schedules after the migration and monitor their execution to ensure they are executing at expected times.

If you programmatically create schedules, please be aware that the API will experience a breaking change at the time of release. You will need to modify any scripts to use the new API structure. We have documented usage of this API on our postman collection - https://www.postman.com/loome-software/workspace/loome

This scheduling functionality will be provided to other modules in future updates.

Major Enhancement to Roles and Permissions

We have significantly reworked roles and permissions within Loome Integrate to provide a more secure experience for teams operating within a tenant.

We now provide security at the project level, allowing you to control what users can see and edit within your tenant.

At the tenant level, we have modified the following roles.

  • Group Admin is now known as Administrator and continues to allow administration of the whole tenant.
  • BI Manager, BI Developer, and BI User have been moved to the project level.
  • The Project Creator role has been added, this allows specified users to create their own projects.
    • This is an optional role, you can opt to only allow Administrators to create new projects.
  • The Consumer role has been added, this provides general access to the tenant.

At the project level, we have added the following roles.

  • Owner – Can perform all roles within a project, including assigning users to roles within a project.
  • Contributor – Assigning a user to this role will allow them to create and edit jobs within a project, this replaces the previous BI Developer role, but limited to that project.
  • Manager - Assigning a user to this role will allow them to run jobs, manage schedules, and view execution history, this replaces the previous BI Manager role, but limited to that project.
  • Member – Allows users to view jobs within a project, this replaces the previous BI User role, but limited to that project.
  • In addition, as an Administrator you can now limit connections, cluster definitions, and file definitions to be available to certain projects only. This is useful for sensitive connections that should not be used by other teams.

Project members can still create their own connections, cluster definitions, and file definitions, and by default these connections will only be available to the project they were created in.

As part of the migration process, all existing BI Developers, BI Managers, and BI Users will be moved to the Project Contributors, Managers, and Users roles respectively in all projects. You can then remove users from projects as you see fit.

Improvements

  • Add Auto Scaling Formula to Azure Batch task, optional field.
    • Value will be considered in Azure Batch task, only if there are no task parameter values for this value.
  • Add Auto Scaling Evaluation to Azure Batch task, optional field.
    • Value will be considered in Azure Batch task, only if there are no task parameter values for this value.
  • Improved UI for user management.

Fixes

  • Fixed a bug where an dependent task may run even if the previous task failed.
  • Fixed an issue where logs sometimes do not load when viewing a job execution.
  • Fixed a bug where a job may sometimes finish early when executing tasks in parallel.

Agent

This update requires agent version - 2023.09.24.1

Please download a new install script and update your agent to use new features, improvements, and fixes.

Improvements

  • Add Auto Scaling Formula to Azure Batch task, optional field.
    • Value will be considered in Azure Batch task, only if there are no task parameter values for this value.
  • Add Auto Scaling Evaluation to Azure Batch task, optional field.
    • Value will be considered in Azure Batch task, only if there are no task parameter values for this value.
  • Added speed improvements to data migrations on SQL Server, Synapse and Azure SQL.

Fixes

  • Fixed an issue where Schema switch may fail due to a table lock.
  • Fixed an issue where an open data reader error may be reported on a data migration that uses Azure Synapse.
  • Fixed an issue where an error relating to a column already having a default bound may be reported on a data migration that uses Azure Synapse.
  • Improved handing of failed tasks when an agent experiences a fatal error such as a VM crash, dropped internet connection, etc.
  • Fixed an issue where an agent running concurrent PowerShell jobs would report that PowerShell core did not exist.
  • Fixed an issue where an agent may enter a non running state, with agent logs stating that it cannot decrypt encryption keys.