Specialized in writing complex SQL queries, creating data pipelines in ADF, and optimizing ETL processes to reduce data load times. Implemented real-time streaming analytics with Azure Stream Analytics and Azure Functions. Managed Kubernetes applications and data migration using Azure Data Factory and Databricks. Developed data pipelines using Apache Kafka for Snowflake integration and private sharing.

Implemented retention policies on AWS and managed decommissioning AWS services after migrating data to Snowflake. Diagnosed issues by pulling data from multiple sources, and developed reports and dashboards using Power BI. Developed and implemented ETL workflows with Talend, enhancing data integration and efficiency. Performed data analysis using AWS S3, Redshift, Snowflake, Python, Spark, and Databricks. Integrated AWS DynamoDB with AWS Lambda for data storage and backup. Optimized data ingestion from various sources to Amazon S3 using a PySpark framework. Worked with DataStage tools and optimized Hive queries for faster processing and efficient job execution.

I developed complex SQL queries for customized data reports and performed data cleaning and preparation using SQL and Excel. I collaborated with the Data Engineering Platform team to deploy and expand Hadoop environments, and designed and implemented AWS data solutions focusing on scalability, performance, and cost optimization. Additionally, I executed MySQL queries from Python using MySQL connector and Db package, and designed, implemented, and operated comprehensive data warehouse systems to optimize data access and batch loading.