Mayur Rao Email and Phone Number

Primarily working on Cloud Migration project where we are moving setup from on premise to Snowflake.As a Data Engineer I am involved in designing and developing Ingestion and Transformation pipelines per business requirement.As part of Ingestion pipeline, we developed Framework to read data from multiple RDBMS/ File based sources via Spark and write to Snowflake refined area.As part of Transformation pipeline we are utilizing Databricks as processing platform on Cloud to read data… Show more Primarily working on Cloud Migration project where we are moving setup from on premise to Snowflake.As a Data Engineer I am involved in designing and developing Ingestion and Transformation pipelines per business requirement.As part of Ingestion pipeline, we developed Framework to read data from multiple RDBMS/ File based sources via Spark and write to Snowflake refined area.As part of Transformation pipeline we are utilizing Databricks as processing platform on Cloud to read data from Snowflake, apply heavy and complex transformation on 20+ Billion dataset and write final dataset back to Snowflake (Domain Objects generation) Show less

Application setup in Hadoop cluster and creating Hive/Impala databases with sentry rolesRequirement gathering from users and Objects creation in Hive with partitioning scheme helpful to users.Used Sqoop tool to import data from variousRDBMS (DB2/Sybase) to HDFS .ETL Development with scripts written in Python.24*7 running jobs to perform incremental load from various RDBMS environment to HadoopFile System.