Sujatha Y Email & Phone Number

United States

• Developed and deployed the project using AWS EC2, EMR, Glue, S3, Lambda, CloudFormation, Elastic Beanstalk, Cloud watch, Elastic search, DMS, SQS, SNS and Amazon Kinesis services to process and store data in Snowflake.• Created automated Databricks workflow notebooks in Python to orchestrate multiple data loads efficiently and Delta Lake tables for metadata storage. • Worked on different types of applications/jobs using PySpark to integrate the data coming from other sources and processed using the Spark data pipelines.• Optimized BigQuery SQL queries by selecting appropriate distribution styles and keys for enhanced query performance.• Involved in developing Spark applications using Spark-SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for Analyzing & transforming the data to uncover insights into the customer usage patterns using Azure & AWS.• Developed Big Query SQL queries with a set of applicable parameters to load data from the HIVE/Presto Stage into the actual HIVE/Presto Target table, often facilitated through Google Cloud SQL.• Loaded data into Spark Data Frames and used Spark-SQL to explore data insights.• Used Spark-SQL to load JSON data and create Schema RDD and loaded it into Hive Tables and handled Structured data using Spark SQL.• Worked on ETL Migration services by developing and deploying AWS Lambda functions for generating a serverless data pipeline which can be written to Glue Catalog and can be queried from Redshift.• Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Spark SQL and Python.

India

• Implementing processes for ingesting data into Delta Lake from various sources, such as streaming data, batch data from on-premises systems, or data from external sources like databases, files, and streaming platforms using tools like Azure Data Factory, Azure Databricks, Azure Stream Analytics, or custom-built data pipelines.• Developed python scripts to execute database SQL queries and scheduled them in Jenkins jobs to show • Status of test case• Verified all the Data load processes developed for fetching data from Big Data system to the Enterprise Data warehouse using SQL queries.• Created HIVE tables for loading and analyzing data, Implemented Partitions, Buckets and developed Hive queries to process the data and generate the data cubes for visualizing. • Created Blob storage to save raw streaming data sent from Streaming Analytics using Kafka to store the final data in Snowflake cloud.• Writing Snow SQL scripts in Snowflake cloud DW to business analysis and reporting.• Familiarity with Hive joins& used HQL for querying the databases eventually leading to complex HiveUDFs. • Worked on the Optimizing storage configurations and created pipelines, partitioning data, and implementing data lifecycle management policies to reduce costs and improve performance using Azure Blob Storage, Azure Data Lake Storage, Azure SQL Database, Azure Cosmos DB, and Azure Synapse Analytics.• Worked on NoSQL databases like HBase, configured MySQL Database to store Hive metadata.• Wrote scripts in UNIX and LINUX environment to verify the remote loading process using secure shell commands.• Used version control tools (GIT & Bitbucket) for source code management and to keep track of the code modifications.

India

• To load the data into Spark RDD's, perform advanced procedures like text analytics and processing using in memory data Computation capabilities of Spark using Scala to generate the Output response. • Deploying and managing the multi-node development and production Hadoop cluster with different Hadoop components (HIVE, PIG, SQOOP, OOZIE, HCATALOG, HBASE, ZOOKEEPER) using Hortonworks Ambari.• Involved in converting Hive/SQL queries into Spark Transformations using Spark RDDs and involved in using SQOOP for importing and exporting data between RDBMS and HDFS.• Created Hive Internal and External tables as per requirements with appropriate static and dynamic partitions intended for efficiency.• Involved in ETL processes developing for fetching data from Hadoop Hive system to the target Data warehouse Oracle using complex SQL queries.• Used AWS EMR to transform and move large amounts of data into and out of other AWS data stores and databases, such as Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB.• Developed real time stream processing application using Spark, Kafka, Scala, and Hive to perform real time data.• Utilized advanced SQL techniques to analyze and profile data from diverse sources, including MSSQL tables and Oracle databases.• Handled importing data from various data sources, performed transformations using Hive, Map Reduce, and loaded data into HDFS using PySpark and Performed complex joins on tables using used various optimization techniques.• Created external tables in Hive, Loaded JSON format log files and ran queries using HiveQL.• Developed in scheduling OOZIE workflow engine to run multiple HIVE jobs.• Created and modified Database objects like Tables, Views, Sequences, Stored procedures, Functions, Packages, Cursors and Triggers as per business requirement.

India

• Involved in converting Hive/SQL queries into Spark Transformations using Spark RDDs and involved in using SQOOP for importing and exporting data between RDBMS and HDFS.• Handled importing data from various data sources, performed transformations using Hive, Map Reduce, and loaded data into HDFS using Spark with Scala.• Used Spark-SQL to load JSON data and create Schema RDD and loaded it into Hive Tables and handled Structured data using Spark SQL.• Used SBT to automate the process of compiling, testing, and packaging Scala code.• Experience in Data Transformations using MR and Hive for different file formats.• Involved in converting HIVE/SQL Queries into transformations.• Performed complex joins on tables in Hive and used various optimization techniques.• Created Hive Internal and External tables as per requirements with appropriate static and dynamic partitions intended for efficiency. • Expert in writing shell scripts and automate with cron job.• Developed stored procedures, functions and triggers to facilitate consistent data entry into the database.• Expert in writing Complex SQL Queries.• Expert in query and Database optimization.• Design and Develop reports using drill down, drill through and drop-down menu option and parameterized and linked options.• Created and modified Database objects like Tables, Views, Sequences, Stored procedures, Functions, Packages, Cursors and Triggers as per business requirement.• Designed and Mapped HIVE external tables on top of HBase tables, HDFS files etc. • Automated importing of data from CSV files into HDFS using OOZIE and HDFS shell commands. • Prepared Functional and Technical Specification documents for new enhancements.• Created Unit Test Case documents.• Coordinated with team for providing requirement clarifications.