Data Architect
New Jersey, United States
• Proficient with container systems like Docker and container orchestration like ECS Fargate.• Managed AWS architecture and automated data pipelines using Terraform and CloudFormation. • Developed automated ETL pipelines using AWS serverless technologies like Lambda, S3, DynamoDB, and SQS.• Monitored system performance and availability using AWS CloudWatch and custom Python scripts.• Worked on Snowpipes to auto ingest hourly files and Snowflake DataSharing to distribute data points for data analytics and BI products to produce critical business insights.• Experienced in Data mining with both structured and unstructured datasets, data acquisition, data validation, predictive modeling, and data visualization.• Created data models on Snowflake DB to improve the efficiency of the automated data pipeline by 600% through parsing and cleansing data during transformation and multi-threading with Python.• Snowflake experience on Warehouses, Stages, Schemas, Functions and Account/User management.• Designed and deployed Data Science machine learning algorithms (XgBoost) by generating modeled features for calculating fraud scores which generated revenue more than $2.5M.• Worked on migrating on-prem Data processing from SQL Server to Cloud using S3, EKS, Glue with Spark which reduced load time by 400% for batch processes.• Proficient with container systems like Docker and container orchestration like ECS Fargate, Kubernetes, also developed Terraform scripts to manage IAAS.• Wrote CI/CD scripts using Gitlab, CFT, ECR and ECS for automating deployments.• Developed Model development and deployment (MDAD) application using real-time data from PostgresDB using Kafka queues and AWS Glue/EMR clusters for running Pyspark jobs.• Built and deployed scalable solutions for Enterprise products using AWS serverless application.• Managed end-to-end encryption of PII/PHI/PRI sensitive datasets.