I conducted thorough assessments of third-party data handling solutions on AWS, ensuring compliance with internal requirements and stakeholder needs. Designed and implemented scalable data pipelines using AWS Glue and Databricks, facilitating real-time data processing and ETL operations. Utilized AWS Lambda and Databricks notebooks for serverless processing tasks, automating workflows to reduce operational overhead. Additionally, I developed and maintained data lakes on Amazon S3, focusing on… Show more I conducted thorough assessments of third-party data handling solutions on AWS, ensuring compliance with internal requirements and stakeholder needs. Designed and implemented scalable data pipelines using AWS Glue and Databricks, facilitating real-time data processing and ETL operations. Utilized AWS Lambda and Databricks notebooks for serverless processing tasks, automating workflows to reduce operational overhead. Additionally, I developed and maintained data lakes on Amazon S3, focusing on data partitioning and storage optimization, while leveraging Apache Spark on Amazon EMR for large-scale processing and performance tuning. My work included building CI/CD pipelines with AWS CodePipeline and CodeBuild, performing data quality checks, and collaborating on data models using Databricks Delta Lake. Furthermore, Developed Spark applications for data analysis, employed data cleansing methods, and adhered to best practices in data engineering, enhancing data reliability and quality across the organization. Show less

Supported "Foundations of Data Science" course, providing tutorials and one-on-one assistance to improve student outcomes. Contributed to course content development, aligning materials with curriculum objectives.Mentored students in machine learning and problem-solving techniques and led presentations on real-world data science projects.

I worked extensively on Google Cloud Dataflow to integrate data from on-prem systems like MySQL and Cassandra with cloud services such as Google Cloud Storage and BigQuery, applying transformations and loading the data back into GCP services. Managed and scheduled resources across clusters using Google Kubernetes Engine (GKE) and monitored Spark clusters through Google Cloud Logging and Dataproc UI. I transitioned log storage from Cassandra to BigQuery, enhancing query performance. I developed… Show more I worked extensively on Google Cloud Dataflow to integrate data from on-prem systems like MySQL and Cassandra with cloud services such as Google Cloud Storage and BigQuery, applying transformations and loading the data back into GCP services. Managed and scheduled resources across clusters using Google Kubernetes Engine (GKE) and monitored Spark clusters through Google Cloud Logging and Dataproc UI. I transitioned log storage from Cassandra to BigQuery, enhancing query performance. I developed data ingestion pipelines on Google Cloud Dataproc using Spark and Dataflow. Additionally, I created dashboards and visualizations in Google Data Studio and Looker, enabling business users and upper management to gain valuable insights. I performed large dataset migrations using Databricks, administered clusters, configured pipelines, and loaded data from Google Cloud Storage to Databricks using Dataflow. Using Google Cloud Functions, I built workflows to schedule and automate batch jobs, and utilized Terraform for pipeline orchestration, Finally, I leveraged Spark Streaming and RDD transformations in Databricks to conduct streaming analytics by ingesting data in mini-batches. Show less

Experienced in analyzing large datasets to uncover trends and patterns, effectively communicating insights through text, charts, and graphs. Skilled in data cleaning, aggregation, and performing missing value imputation to enhance data quality. Demonstrated ability to assess data management metrics and provide recommendations for enterprise-wide data improvements. Successfully led recruitment efforts and developed strategic alliances to optimize team capabilities. Proficient in handling… Show more Experienced in analyzing large datasets to uncover trends and patterns, effectively communicating insights through text, charts, and graphs. Skilled in data cleaning, aggregation, and performing missing value imputation to enhance data quality. Demonstrated ability to assess data management metrics and provide recommendations for enterprise-wide data improvements. Successfully led recruitment efforts and developed strategic alliances to optimize team capabilities. Proficient in handling large-scale datasets, such as customer credit attributes from TransUnion, to drive actionable business insights. Show less