Optimized Airflow DAGs data pipelines for data warehousing to consolidate semi-structured data from various data sources (APIs, MySQL, Oracle) into AWS Redshift for enterprise use casesPerformed root cause analysis on internal and external data and processes to optimize existing business practicesImplemented with data streaming solutions with AWS Kinesis and AWS Kafka (MSK) for big data warehousingConstructed ETL pipeline consolidating ERP systems LiveChat, LiveEngage, and Oracle DB data for forecasting business labor requirementsInteract closely with stakeholders to determine analytics needs and translate those into efficient and scalable data processesBuild and maintain data models with DBT and writing the YAML and SQL files for rendering DAGs in AirflowWrite DBT macros to simplify data models and improve scalabilityCreate DAG factories for our SFTP ELT pipelines drastically saving time in creating new pipelines

Implemented role-based access controls (RBAC) and data masking within Snowflake to maintain strict data security standards and ensure compliance with industry regulationsDeveloped data quality validation tests to ensure accuracy and reliability, reducing pipeline failure rates by 50%Spark pipelines loading client manufacturing and sales data into Google Cloud Storage, enabling efficient data querying for report optimization of 33%Automated ETL pipelines using Spark Python to load raw KLARF/EDX wafer files into Snowflake tables, significantly improving data processing speed by 40%Enhanced data processing efficiency by leveraging Snowflake's scalable architecture to handle large-scale data, optimizing storage and query performance for complex datasetsSuccessfully integrated Snowflake with GCP services (e.g., Google Cloud Storage and Cloud Dataflow) to create a seamless data pipeline, enabling efficient data querying and real-time insightsLed the migration of legacy data warehouse systems to Snowflake, ensuring data integrity, minimizing downtime, and enhancing data accessibility across the organizationCollaborated frequently with India teams for project management on IST and PST time zones (+12.5 hours)Data streaming solutions with Apache Kafka for event-driven ETL enhancing data ingestion rates by 40%Statistical tests including ANOVA, chi-squared, t-test, and hypothesis tests model building for forecasting reports

- Taught fundamental machine learning concepts in linear algebra, calculus, and statisticsInteractive instruction using Scikit-Learn, Tensorflow, Pandas, and Numpy for regression, classification, and clustering techniques - Instructed with Jupyter notebooks to teach fundamental machine learning algorithms such as linear regression, LASSO, Ridge, gradient boosting, XGBoost, random forest, naive bayes, SVM, KNN, K-Means- Taugh data visualization methods like histograms, box plots, and scatter plots for data preperation

Developed Python OpenCV apps to analyze silicon wafer chips, improving defect detection accuracy by 20%Designed a gradient boosting regression model to project annual sales figures for an automotive clientConstructed multi-node clusters and data processing using Spark, optimizing job performance by 30%-60%Created data partitioning and indexing strategies, improving query performance and efficiency by up to 60%Designed and implemented a Vehicle Fleet Management System using Django, enabling efficient tracking of vehicle status, driver assignments, maintenance records, and trip history, streamlining fleet operations and reducing downtime by 25%.Integrated RESTful APIs using Django REST Framework to enable cross-platform data access and exchange, supporting integrations with IoT devices and vehicle telemetry systems for real-time tracking and reportingCreated an admin interface using Django Admin for easy management of fleet assets, driver profiles, maintenance logs, and trip scheduling, reducing manual administrative work by 40%Optimized database queries for handling large datasets, ensuring fast retrieval times for complex filtering and reporting operations on vehicle performance and maintenance historyDeployed the application using Docker and AWS Elastic Beanstalk, ensuring scalable, resilient infrastructure capable of handling growing fleet data and concurrent user access.