Electricity North West is the distribution network operator for the North West of England. It operates 44,000km of overhead lines and underground cables to take power from the National Grid to 2.4m properties across the region.

Moving data from legacy systems into more modern databases is a challenging process. Stakeholders still need access to records to make key business decisions, but having a complete picture is difficult during the transition period as both databases are “active”.

Staff update records in both systems regularly. To ensure a “complete picture”, any changes made must be duplicated in both databases.

This complicated task tends to fall on the heads of already-strained data teams, who have to juggle their daily duties with ensuring data quality is maintained across both systems.

The Challenge

Electricity North West found themselves in this situation during their move from their legacy Oracle-based asset database (Ellipse) to a more modern API-based system (Chime). As the data held in these systems contained work orders that the organisation was actively updating, it was key that changes in either the Ellipse or Chime database were duplicated into the other.

The team needed a tool capable of moving this high volume of data, as well as an automated process that identified changes and updated the corresponding record in the other asset management system.

The Solution

We proposed an FME-based solution, as Electricity North West already utilised the software to process high volumes of data in other areas of the business.

We began by creating a prototype of our solution in FME Desktop. The drag and drop nature of FME made the design of the initial solution quick to put together and easy for Electricity North West to understand and feedback on. Once agreed, Electricity North West created a range of user scenarios that we could use to optimise the final solution.

An FME Server endpoint “listened” to both of the databases to detect when a change was made. This kicked off a workspace that pulled the data from the updated record, transformed it into the correct schema and presented it to the opposite system either by APIs or file based bulk loaders. These automated processes ran on daily schedules to ensure that all records changed were updated and that the databases were aligned at all times.

To ensure the solution is working correctly, we also used FME to create a log of any changes to the solution processes. This log provided visibility to users that may not be familiar with FME itself but still need to monitor changes in the systems.

 

The Benefits

Rather than investing in new ETL/DIP (Extract Transform and Load / Data Integration Platform) software, Electricity North West leveraged an existing investment to create a solution that not only saved time but maintained a high standard of asset data.

Automating all aspects of the updates means that teams no longer had to spend time performing duplicate tasks and could focus on their day-to-day workload.

This also minimised the need for manual intervention in the data transfer process and removed it completely where possible, which lowered the risk of errors being introduced to their data.