Data Engineer
Current Implemented data validation processes, reducing AI model errors by 25%. Collaborated on feature engineering and automated feature extraction. Enhanced AI model training by integrating diverse data sources. Developed customer segmentation engines and predictive models with Power BI and MSBI. Utilized Azure Blob Storage’s geo-redundant storage for high availability. Implemented Redshift UDFs for custom data processing and used Snowflake data sharing for secure collaboration. Designed RBAC for Azure Synapse resources. Maintained Hadoop and Spark clusters for reliable infrastructure. Built interactive visualizations in Plotly and used AWS Glue logging for troubleshooting. Managed ETL code deployment and Azure DevOps service connections. Analyzed location-based data with BigQuery geospatial functions. Configured Azure Data Factory pipelines for performance and processing. Leveraged Azure Cosmos DB's MongoDB API for integration and Power BI with Dynamics 365 for analytics. Set up data lake integration with Azure Firewall for security and documented transformation processes. Integrated Azure Purview with data quality tools for profiling. Engineered encryption key vault rotation strategies and developed Python apps for retail. Created visualizations in QlikView with JavaScript and HTML. Developed data governance forums and maintained lineage reports. Utilized Power BI with Azure for storage and processing, and optimized Spark in Azure Data Bricks. Deployed event-driven architectures with Apache Kafka for streaming and designed data pipelines for BI tools like Tableau. Managed BI solutions for self-service analytics and worked with AWS Data Pipeline for data loads into Redshift. Used AWS EMR for data transformation and moving, enforced integrity with DataStage, and monitored ADF data sets. Integrated Lambda with SQS and DynamoDB with step functions for message updates.