Ninad Thaker Email & Phone Number

Dublin, Ie

Dublin, County Dublin, Ireland

• Migrated Jenkins as Code pipelines to Git Actions workflow using Python and GitHub cli• Conducted interviews for engineering position • Modified and improved core business logic to segregate data at record level for enforcement of encryption/ masking policies• Designed Airflow dags with dynamic task mapping containing controlled concurrent execution of tasks to allow data movement via Apache Spark with optimal resources allocation• Worked on Data Enrichment topics• Designed customisable task level notifications for Airflow tasks• Designed and proposed a metadata driven on prem Data Engineering Architecture allowing a new project to deliver ML models and reports• Implemented Motif & GraphFrame based - Spark code allowing tracking of an individual record's tenure with Arbitrary Start and End points having dynamic edges in between• Integrated JFrog X-ray in CI/CD for Auditing Vulnerabilities and Prohibited License scanning• Helped designing Data Quality Enforcement and Rule Management with Amazon Deequ by using Persistent Delta Table for Scalability and Dynamism on a DeltaLake Medallion Architecture with Azure Databricks and Snowflake• Implemented Data Aware Dataset Airflow DAGs• Designed custom functionality to fetch dynamic number of tasks logs and error via Databricks Hook on Airflow using Databricks API• Architected part of custom Feature Store containing change data feed, liquid clustering, delta lake and snowflake merge rule sets with dynamic data masking along with Lakehouse Quality Monitoring • Architected core analytical pipeline with automatic cluster and warehouse selection for growing dimensional data model• Designed on demand self service view creation solution with Airflow, Databricks and Snowflake for PHI and NON - PHI data from multiple Snowflake Storage Units• Worked on POC for implementing External Iceberg Table stored in ADLS Gen V2 via Databricks and read from Snowflake

Dublin, County Dublin, Ireland

• Worked on PHI/ PII data with HIPAA and GDPR regulations ensuring privacy by applying Data de-identification • Built data ingestion logic using Scala that cleans, transform, and aggregate data from disparate sources such as SQL server, CSV/text files, and Teradata through Apache Spark• Designed custom Airflow DAG Template to generate DAGs without writing single line of code by using YAML as configuration file allowing smooth onboarding for the Data Science and Business Intelligence teams, and faster updates across all DAGs to orchestrate automated data ingestion.• Designed, improved, and maintained a modular Scala-based common ingestion framework to reduce development time & increased the amount of total ingestion • Using Delta table and PySpark as the backend, designed Auditing of Hive tables to help reduce the amount of time spent on common queries by keeping track of daily ingested data and their SCDs • Used agile software development processes to make iterative improvements to the backend systems• Worked closely with our Data Science team and Business Intelligence team to help build complex algorithms that provide unique insights into our data• POC Participated in the design of delta architecture-based common framework to ingest data from on-premise Hive/ SQL/ NAS Servers using ADF and publishing data into the Snowflake DW deployed on Azure• Worked on GitHub Enterprise Server to GitHub Enterprise Cloud migration• Certified Medicare & Retirement (M&R) Foundation

Master Of Science - Ms, Data And Computational Science

Bachelor Of Technology - Btech, Computer Science

Activities and Societies: Intern: Python - Django Project Management and Monitoring System Intern: Python - Image Processing and Custom.