As a Data Engineer, I have a proven track record in transforming data ecosystems using key technologies like DBT, Snowflake, and Azure Data Factory. My expertise lies in designing robust data pipelines to ensure seamless data flow and drive insightful decision-making.Project Objective:Led a pivotal project to centralize data from transactional systems and third-party databases into a unified warehouse. The goal was to expedite analytical report development, providing critical… Show more As a Data Engineer, I have a proven track record in transforming data ecosystems using key technologies like DBT, Snowflake, and Azure Data Factory. My expertise lies in designing robust data pipelines to ensure seamless data flow and drive insightful decision-making.Project Objective:Led a pivotal project to centralize data from transactional systems and third-party databases into a unified warehouse. The goal was to expedite analytical report development, providing critical business insights for quicker decision-making.Key Contributions:Azure Data Factory Implementation:Designed and implemented Azure Data Factory pipelines for smooth data movement across diverse databases and file systems.Snowflake Data Warehouse Integration:Orchestrated data ingestion into Snowflake, adopting the Medallion Architecture for direct reflection of source data.DBT Transformation Expertise:Leveraged DBT for architecture design and applied transformations, utilizing its open-source flexibility.Apache Airflow Orchestration:Implemented Aapache Airflow for efficient orchestration of DBT pipelines across various environments.Extract Generation and Storage:Used Apache Airflow to generate extracts, storing them in Azure Blob Storage and SFTP servers for easy accessibility.Results:The project streamlined data processes, optimizing workflows for unified reporting and analysis. Implemented solutions enhanced business intelligence capabilities, empowering quicker decision-making.Excited about contributing my expertise to new challenges. Let's discuss how my skills align with your organization's objectives. Show less

During the tenure of this project below are my key contributions in projects.Migration Expertise: Led the flawless migration of on-premises databases to Azure SQL databases, demonstrating a commitment to enhanced scalability and superior performance in cloud environments.Streamlined Data Processing: Engineered robust pipelines for efficient data transfer from transactional databases to reporting databases, addressing the unique requirements of various teams within the… Show more During the tenure of this project below are my key contributions in projects.Migration Expertise: Led the flawless migration of on-premises databases to Azure SQL databases, demonstrating a commitment to enhanced scalability and superior performance in cloud environments.Streamlined Data Processing: Engineered robust pipelines for efficient data transfer from transactional databases to reporting databases, addressing the unique requirements of various teams within the organization.Change Data Capture (CDC) Mastery: Implemented CDC processes from transactional databases, guaranteeing both real-time and historical data accuracy critical for informed business insights. Engineered optimized stored procedures for code transparency, ensuring faster and reliable data processing with minimized computing costs.Business Requirements Gathering: Collaborated closely with business stakeholders to gather requirements for creating data models and implementing data visualizations for the MIS team, aligning data solutions with strategic business needs.Third-Party Data Integration: Developed extracts for third-party vendors, facilitating seamless data transfer to Azure Blob Storage accounts and SFTP servers, showcasing adaptability to diverse data integration scenarios.Skills:Azure SynapseAzure Data FactoryAzure Blob StorageChange Data Capture (CDC)ETL OptimizationData Pipeline DesignCollaboration with Cross-functional Teams Show less

Migrated Existing on premises database to Azure Synapse dedicated sql pool database and existing ETL which were developed on Pentaho Spoon ETL and were running on on premise servers to Azure Data Factory (ADF) for RBL Bank.Implemented Change Data Capture (CDC) process to capture data changes from finacle systems to ensure accurate and up-to-date data as well as historical data.Collaborating with MIS team members and Finance team members to produce extract as per automated or adhoc… Show more Migrated Existing on premises database to Azure Synapse dedicated sql pool database and existing ETL which were developed on Pentaho Spoon ETL and were running on on premise servers to Azure Data Factory (ADF) for RBL Bank.Implemented Change Data Capture (CDC) process to capture data changes from finacle systems to ensure accurate and up-to-date data as well as historical data.Collaborating with MIS team members and Finance team members to produce extract as per automated or adhoc basis as per the requirement.Utilized Azure Data Factory, to design and develop data pipelines to efficiently extract data from various sources, including data file systems, on-premises databases, and cloud databases. Show less