Sujith Vatsavai Email & Phone Number

United States

Amherst, New York, United States

• Migrated the Hive metastore catalog to Unity Catalog in Databricks to achieve 100% unified access control. Utilized Auto Loader and Delta Live Tables to process near-real-time (NRT) data.• Orchestrated ETL and ELT pipelines in Azure Data Factory and performed transformation in databricks notebooks later ingest data into delta lake and AzureSql,ADLS.• Developed Databricks notebooks using PySpark and SQL to process a 50GB large dataset, load data into views by abstracting the Sql ,Delta tables, and integrated with PowerBI to generate dashboard.• Associated Azure Logic Apps and Azure Functions in Python with Azure Monitor for real-time data pipeline monitoring and alerting, resulting in a 30% improvement in issue resolution time.• Scheduled Workflows with Apache airflow to perform complex transformation using python Dags and united with ADF to monitor entire pipeline resulting 99% issue resolution.• Developed REST Api’s using Python,FastAPI, SQLAlchemy, NumPy,Pandas resulting in a 25% reduction in response time. Deployed applications on Azure VM using Azure DevOps,Azure Kubermetes.• Applied Snowflake features like Snowpipe and stages to process 10GB of data using DBT scripts for transformations every day in real-time and ingest the data into Snowflake tables.

Chennai, Tamil Nadu, India

• Utilized Agile methodology with Azure Boards for sprint planning, ensuring complete alignment with sprint deadlines for a team of 4 individuals and collaborated with cross-functional teams.• Generated SQL Server stored procedures, functions, views, triggers and complex queries using both sql and T-sql improved 30% performance.• Designed real-time data pipelines in Azure Synapse triggered by Event Hubs and loaded data into Delta Lake and leveraged Delta Live Tables to load data into views, enabling real-time updates to PowerBI dashboards and improving business decision-making by 30%.• Created Spark applications using Pyspark and Spark-SQL programming with 30% improvement in data extraction, and aggregation from multiple file formats(JSON, Parquet, Avro, CSV, XML).• Operationalized CI/CD pipelines in Azure DevOps utilizing Docker to migrate Azure Synapse, Databricks, and Azure SQL resources to higher environments, reducing processing time by 20%.• Efficiently migrated data from SQL Server, Oracle, and MySQL to Azure Synapse Analytics and Azure Data Lake, achieving a 30% reduction in data latency.

India

• Implemented Spark Streaming applications to transform data in real time. Realized a 20% decrease in processing time, ingesting data from Kafka in to Cassandra for real-time analytics in Kinesis.• Created RESTful APIs by leveraging AWS services (API Gateway, Lambda, Route 53, IAM, S3, CloudWatch) and load the StepFunctions outputs to DynamoDB.• Built microservices using Django by creating models, views, serializers to feed data to a React application and integrated with Kafka as a messaging queue, decreasing response time by 30%.• Crafted ETL jobs in Glue Studio, applying diverse transformations in EMR, and ingesting data into S3, Redshift, and RDS reduced 40% completion time.• Proficiency in configuring, working with big data technologies achieving a 20% progress in performance using Hadoop, Sqoop, Spark,Hive,MapReduce and HBase on AWS EMR clusters.

Bachelor Of Technology, Computer Science And Engineering

Master Of Science - Ms, Computer Science