Transforming The Data Architecture Using Azure Data Services
Read how we helped a customer upgrade their legacy data architecture by implementing Azure Data Services
Connecticut, USA
Financial Services
11-50 employees
The Customer
They are a private equity firm that invests in growth-stage software, healthcare, and technology-enabled services companies. They focus on innovative business models and technologies, providing capital, strategic guidance, and operational support. The firm collaborates closely with management teams to drive value creation through growth initiatives and strategic acquisitions. They are known for their hands-on partnership approach.
The Problem
- The client had a .NET application for their ETL process to import data from multiple sources. Anytime a change is required, a whole development cycle was executed to incorporate the change. Data was coming in from multiple companies and from different sources and they wanted an option to set the priority source which was not available in the existing system.
Establish a canonical data model to support our application.
Create a 'golden universe' of companies identified by primary ID based on domain.com
Map out the origin of data fields from various sources for each domain.com
Enable data sourcing via real-time APIs, Snowflake, and file inputs to populate the golden universe.
Implement an abstraction layer separating the application from underlying data sources.
Ensure rapid, continuous updates to the golden universe, avoiding batch processes.
Incorporate a 'refresh from sources' feature for immediate data updates and employ background updates based on data staleness or changes from providers.
What Solution Did We Propose
We propose to automate the process using Azure Data services where we would leverage Azure Data Factory to perform all data pipelining and Azure Databricks to perform all data transformations
Folio3 deployed an end-to-end solution to transform the data processes for the client. Our solution helped them speed up their data processing and management by eliminating the need for manually running development cycles and automating the process using Azure Data Services.
Raw data upload
Developed ETL pipelines using Azure data factory
Performed Data Ingestion
Carried our data transformation using Azure Databricks
Data Lake and azure SQL database for data storage
Azure data vault to store credentials and keys
Deployment pipelines were setup using Azure Devops
Technologies Involved In This Case
Azure SQL
Azure Storage Account
Azure Data Factory
Azure Key Vault
Azure DevOps
Azure Databricks
Take A Seamless Cloud Ride With Us!
Get the power to boost your business, innovation, and growth.
