Senior Data Scientist
CurrentBuilt a NLPCoP product (NLP community of practice), groups of AI enthusiasts who share a concern or passion for NLP using AI and learn how to do it better as they interact regularly. Designed Model for Risk Audit and Inspection Category Reclassification to evaluate the possibility of observations into different categories, classify the observational data using category definition.Designed Model for Semantic Textual Similarity to Finding the similar deviation based on the historical deviations.Designed Model for Clinical Trial LPLV-DBL Prediction for different Phases to predict the time duration from LPLV to DBL for a clinical trial.Implemented Apache Airflow for authoring, scheduling, and monitoring Data Pipelines.Scheduled Airflow DAGs to run multiple Hive, which independently run with time and data availability.Involved in optimizing Snowflake cost optimization initiatives with data models and efficient queries.Proficient in utilizing Apache Spark within Databricks for large-scale data processing.Worked on Snowflake environment to remove redundancy and load real time data from various data sources into HDFS using Kafka.Developed Python-based API (RESTful Web Service) to track revenue and perform revenue analysis.Worked with Tableau for generating reports and created Tableau dashboards, pie charts, and heat maps according to the business requirements.Designed and Developed Scala workflows for data pull from cloud-based systems and applying transformations on it.Involved in handling large datasets using Partitions, Spark in-memory capabilities, Broadcasts in Spark, Effective & efficient Joins, Transformations and other during ingestion process itself.Designed Model for NLP Knowledge discovery to get the insights from YOF response data to understand the diversity of responses across various countries/roles, key actionable sentiments & themes.Context based mining of each questions on Context/Semantic based sentiment analysis using BERT.