A speedy listen, generated using Google Notebook LM.
Say you have a data process running in Azure and now you need to run this identical process in AWS. Azure Data Factory can connect to AWS. AWS Glue can connect to Azure. However, their capability is limited because they’re designed to run processes in their native cloud.
Copying the logic from Azure Data Factory to AWS Glue isn’t easy either. Each has its own operating model so you can’t just lift and shift the processes over.
Even if you introduce an orchestration layer, you still have to set up and manage both Azure Data Factory and AWS Glue.
Unfortunately, all this means that when a process needs to move, it often has to be rebuilt. Imagine if that process you spent so much time building just worked anywhere.
FME approaches the problem differently because it has two universal core components: Workspaces and Engines. Workspaces consistently define the data process. Engines deliver the data processing anywhere you need it to.
If you have an FME Workspace, you can send it to an Engine in Azure, and it will run that defined process. You can send that same Workspace to an Engine in AWS, and it’ll run the same defined process. You can send that Workspace to an Engine on-premises and, yes, it’ll run the same process.
The workspace file is portable too. You can upload it to FME Flow, where it can be run by anyone with access. You can share it as an email attachment and it’ll still work. Heck, you can WhatsApp the thing and the process logic would still be intact.
In hybrid cloud, consistency stops complexity multiplying. With FME, the Workspace becomes that consistent unit of work. Build it once, run it where it needs to, and let the Engine do the processing.
Of course, environment-specific setup around credentials, connectivity, and compliance changes but your core process logic doesn’t.
That means there’s less duplication, less rework, less platform-specific support, and fewer differences creeping into processes that are doing the same thing. So, stop doing the same thing again and again and again.
Insightful, shareable, available here.
Why you shouldn’t have to order your pint differently in the The Azure Arms, The S3 & Barrel, and The Lambda Lounge.
Imagine your data pipeline is a busy motorway carrying thousands of vehicles from one place to another. Based in a motorway service station, you’ve been given the task to check all the red cars hurtling down the road while minimising the disruption to the rest of the traffic. How would you do it?
Read MoreCloud was supposed to make cost simpler, but as we’ve developed hybrid infrastructures and incorporated more tools it’s become exponentially more difficult to manage.
Read MoreYour environment didn’t appear overnight. It evolved with every platform decision, every architecture review, and every workload that moved to the cloud. The problem is that most data tools were built for a specific moment in that journey.
Read MoreIf our environments are in flux, the tools underneath them need to cope with that and not become another constraint. So rather than choosing a tool for one environment, we should be looking for the qualities that let it keep working across many.
Read MoreWe’ve focused heavily on control, and in doing so, we’re starting to lose sight of the ownership of our data.
Read MoreMastery comes from using the same tools often, not constantly switching between new ones. The pressure is on IT teams to do the opposite.
Read MoreCloud has fundamentally changed how organisations store and process data. What we’re calling “cloud” isn’t just about location, it’s actually a set of operating models that shape cost structures, skill requirements, and how easily a business can adapt over time.
Read MoreIf you still think of FME as a desktop tool, you’re already behind. The cloud-first features make deployment, scaling, and cost management easier, smarter, and more efficient.
Read More