• Designed and implemented scalable data ingestion pipelines using Azure Data Factory, ingesting data fromvarious sources such as SQL databases, CSV files, and REST API’s• Developed data processing workflows using Azure Databricks, leveraging Spark for distributed dataprocessing and transformation tasks• Ensured data quality and integrity by performing data validation, cleansing, and transformation operationsusing Azure Data Factory and Databricks.• Designed and implemented a cloud-based data warehouse solution using Snowflake on Azure, leveraging itsscalability and performance capabilities• Built the data pipeline using Azure Service like Data Factory to load the data from the Legacy SQL server to AzureData Base using Data Factories, API Gateway Services, SSIS Packages, Talend Jobs and Python• Built the trigger-based Mechanism to reduce the cost of different resources like Web Job and Data Factoriesusing Azure Logic Apps and Functions• Performed ETL using Azure Data Bricks. Migrated on-premises Oracle ETL process to Azure SynapseAnalytics• Created and optimized Snowflake schemas, tables, and views to support efficient data storage and retrievalfor analytics and reporting purposes.• Collaborated with data analysts and business stakeholders to understand their requirements andimplemented appropriate data models and structures in Snowflake• Developed and optimized Spark jobs to perform data transformations, aggregations, and machine learningtasks on big data sets.• Leveraged Azure Synapse Analytics to integrate big data processing and analytics capabilities, enablingseamless data exploration and insights generation.• Configured event-based triggers and scheduling mechanisms to automate data pipelines and workflows.• Implemented data lineage and metadata management solutions to track and monitor data flow andtransformations.

•Implemented end-to-end data pipelines using Azure Data Factory to extract, transform, and load (ETL) data from diverse sources into Snowflake•Designed and implemented data processing workflows using Azure Databricks, leveraging Spark for large-scale data transformations.•Built scalable and optimized Snowflake schemas, tables, and views to support complex analytics queries and reporting requirements.•Developed data ingestion pipelines using Azure Event Hubs and Azure Functions to enable real-time data streaming into Snowflake.•Leveraged Azure Data Lake Storage as a data lake for storing raw and processed data, implementing data partitioning and data retention strategies.•Utilized Azure Blob Storage for efficient storage and retrieval of data files, implementing compression and encryption techniques to optimize storage costs and data security.•Integrated Azure Data Factory with Azure Logic Apps for orchestrating complex data workflows and triggering actions based on specific events.•Implemented data governance practices and data quality checks using Azure Data Factory and Snowflake, ensuring data accuracy and consistency.•Implemented data replication and synchronization strategies between Snowflake and other data platforms using Azure Data Factory and Change Data Capture techniques.•Developed and deployed Azure Functions for data preprocessing, data enrichment, and data validation tasks in data pipelines.•Implemented advanced analytics and machine learning workflows using Azure Machine Learning and Snowflake, enabling predictive analytics and data-driven insights.•Designed and implemented data archiving and data retention strategies using Azure Blob Storage and Snowflake's Time Travel feature.•Developed custom monitoring and alerting solutions using Azure Monitor and Snowflake Query Performance Monitoring (QPM) for proactive identification and resolution of performance issues.

•Proficient in deploying and managing Azure VMs for scalable and flexible computing solutions.•Extensive experience with Azure Blob Storage for efficient and secure data storage, retrieval, and management•Implemented Azure AD for seamless user authentication, authorization, and identity management in cloud applications.•Developed and deployed web applications using Azure App Services, ensuring high availability and scalability.•Proficient in designing and implementing scalable and flexible data models using Azure Cosmos DB, accommodating diverse data types and optimizing for high-performance NoSQL storage.•Proficient in writing and optimizing complex T-SQL queries for efficient data retrieval, aggregation, and manipulation, ensuring optimal database performance.•Managed Azure SQL Database for robust and scalable relational database solutions, optimizing performance and security.•Conducted performance tuning in Snowflake, optimizing SQL queries, indexing, and clustering to enhance data processing speed and efficiency.•Implemented caching strategies and materialized views for improved query performance.•Implemented robust security measures in Snowflake, including role-based access controls, encryption, and audit logging.•Ensured compliance with industry standards and data protection regulations in the handling of sensitive data within Snowflake.•Conducted data profiling, validation, and reconciliation to ensure data accuracy during migration processes.•Conducted cost analyses and recommendations for right-sizing Snowflake clusters and storage.

•Designed and implemented scalable and efficient data processing pipelines using technologies such as Apache Hadoop and Apache Spark.•Conducted in-depth data analysis to extract valuable insights and support data-driven decision-making processes.•Developed and maintained large-scale distributed databases, optimizing performance and ensuring data integrity.•Implemented data warehousing solutions for efficient storage, retrieval, and analysis of structured and unstructured data.•Proficient in programming languages such as Java, Python, and Scala for developing robust data applications.•Created and optimized scripts for data extraction, transformation, and loading (ETL) processes.•Extensive experience with big data technologies, including Apache Hadoop ecosystem components (HDFS, MapReduce) and Apache Spark for large-scale data processing.•Utilized tools like Apache Hive and Apache Pig for data transformation and analysis.•Designed and implemented data models to support business requirements and ensure effective data organization.•Developed and maintained data schemas and structures for optimal performance and scalability.•Implemented real-time data processing solutions using technologies such as Apache Kafka for streaming data ingestion.•Ensured the availability and reliability of real-time data streams for immediate business insights.•Collaborated with cross-functional teams to understand business requirements and translate them into effective data solutions.•Communicated complex technical concepts to non-technical stakeholders to facilitate decision-making.•Conducted performance tuning and optimization of big data applications to enhance overall system efficiency.

Designed, developed, and maintained end-to-end ETL processes, ensuring seamless data extraction, transformation, and loading from source to target systems.Orchestrated data workflows to support business intelligence, analytics, and reporting requirements.Demonstrated expertise in ETL tools such as Informatica, Talend, or Apache NiFi, utilizing their functionalities for data integration and transformation.Developed and optimized ETL jobs and workflows to align with business objectives and data quality standards.Created and maintained data models and mappings, defining the transformation logic to ensure accurate and consistent data representation.Collaborated with data architects to design efficient and scalable data structures for ETL processes.Conducted performance tuning activities to optimize ETL job execution times, resource utilization, and overall system efficiency.Implemented indexing, partitioning, and caching strategies to enhance ETL process performance.Established and implemented data quality checks within ETL workflows to identify and address anomalies or discrepancies in the data.Collaborated with data stewards and business users to define data quality rules and metrics.Integrated ETL processes with diverse source systems, including databases, APIs, flat files, and cloud-based platforms.Ensured compatibility and seamless data transfer between source and target systems.Implemented robust error-handling mechanisms to capture and log errors during ETL processing, facilitating troubleshooting and issue resolution.Conducted root cause analysis for ETL job failures and implemented preventive measures.Implemented Change Data Capture techniques to capture and process only the changed or updated data, minimizing processing time and resources.Utilized CDC to support near-real-time data integration and synchronization.Created comprehensive documentation for ETL processes, including design specifications, data lineage, and operational procedures.