Created S3 tables using Lambda Functions and AWS Glue, with Python for smooth execution.Collaborated with the team to build a framework for generating daily reports and data extracts from enterprise data, making use of automation for efficiency.Set up a robust monitoring system, including Lambda functions, Glue Jobs, and EC2 hosts, integrated with CloudWatch for real-time performance tracking, alerts, and detailed logs.Employed AWS Glue for data transformation, validation, and cleansing, ensuring high data quality.Configured AWS Glue, EC2, and S3 services efficiently using Python's Boto3 libraries, optimizing our data infrastructure.Orchestrated end-to-end ETL processes, involving Lambda Functions and AWS Glue DataBrew, streamlining data pipeline development.Improved ETL services by deploying AWS Lambda functions, creating a serverless data pipeline that integrates with the Glue Catalog and allows querying through Athena for simplified data analysis.

Created data pipelines to handle structured and unstructured data from various sources, using ETL tools like Azure Data Factory, SSIS, and Mosaic.Developed SQL components like Stored Procedures, Indexes, Triggers, and Views in Azure SQL Database to maintain data quality and enable data sharing via APIs.Built Spark applications with PySpark and Spark-SQL to extract, transform, and analyze customer usage data from diverse file formats.Led the implementation of Azure Data Factory (ADF) pipelines, streamlining the migration of elevator data from on-premises county systems to Azure Blob Storage. This effort reduced onboarding time by 30%.Successfully conducted a Proof of Concept (POC) in Snowflake and migrated data and processes from Azure SQL Database, resulting in cost savings of 20% and faster query processing.Optimized over 50 ETL jobs, reducing failure rates by 30% through identifying and addressing common issues and bottlenecks.Executed Proof of Concepts (PoCs) for SOAP and REST APIs to access analytical data, using tools like MS SQL and Tableau for data analysis and visualization in support of data analytics efforts.