Susmitha R Email & Phone Number

Texas, United States

● Designed, developed, and tested dimensional data models using Star and Snowflake schemas by the Kimball methodology.● I design and manage scalable data warehouses using Snowflake, ensuring efficient data storage and retrieval. ● Developed and maintain complex DBT models that streamline data transformation and enhance data quality.● Integrated DBT with CI/CD pipelines to automate deployments and ensure consistent, reliable data delivery.● Managed AWS tools, including CloudWatch and CloudTrail, to monitor and track cloud resources and activities.● Implemented user versioning in S3 buckets for highly sensitive data storage.● Integrated AWS DynamoDB with AWS Lambda to manage data storage and backup through DynamoDB streams.● Installed and configured Apache Airflow for AWS S3 buckets and created Directed Acyclic Graphs (DAGs) for workflow automation.● Developed scripts for automating data ingestion processes using PySpark and Scala, connecting to various sources, including APIs, AWS S3, Teradata, and Redshift.● Migrated on-prem Informatica ETL processes to AWS cloud and Snowflake.● Employed Spark streaming to divide streaming data into batches and process them in the Spark engine.● Wrote Spark applications for data validation, cleansing, transformation, custom aggregation, and analysis, making data available to data scientists.● Proficient in building Snowpipe and experienced in Data Sharing within the Snowflake Database, Schema, and Table structures.● Migrated on-prem Informatica ETL processes to AWS cloud and Snowflake.● Expertise in migrating existing databases from on-premises to AWS Redshift using various AWS services.● Created, modified, and executed DDL in AWS Redshift and Snowflake tables for data loading

● Created ETL mapping document and ETL design templates for the development team.● Created external tables on top of S3 data which can be queried using AWS services.● Developed ETL pipelines in and out of the data warehouse using a combination of Python and Snowflakes Snows Writing SQL queries against Snowflake.● Utilized Matillion to design and implement complex ETL workflows that streamline data integration and transformation processes.● leveraged Matillion's intuitive interface and powerful features to optimize data workflows, ensuring high performance and reliability. Additionally, implemented best practices for error handling, logging, and monitoring to maintain robust and resilient ETL processes. By automating repetitive tasks and optimizing resource usage, and enhance the efficiency and scalability of data operations.● Utilized Azure Machine Learning to develop predictive models and apply advanced analytics techniques, empowering data-driven decision-making and optimizing business processes. ● Conducted ETL testing, including extracting data from databases, transforming data, and uploading it to data warehouse servers using SQL and NoSQL. ● Proficient in designing and developing multidimensional and tabular data models using SQL Server Analysis Services (SSAS).● Worked on developing Pig scripts to create the relationship between the data present in the Hadoop cluster.● Performed advanced procedures like text analytics and processing using the in-memory computing capabilities of Spark using Python and Scala.● Worked on implementing data lake and was responsible for data management in Data Lake.● Developed Ruby Script to map the data to the production environment.● Experience in analyzing data using Hive, HBase and custom Map Reduce program.● Developed Hive UDFs and Pig UDFs using Python script.

● Experience in architecting, designing, installing, configuring, and managing Apache Hadoop Clusters, MapR, and Hortonworks & Cloudera Hadoop Distribution.● Developed ETL data pipelines using Spark, Spark streaming, and Scala.● Building Real-Time Data pipelines that code modularization, with dynamic configurations to handle multiple data transfer requests without making the production job changes.● Loaded data from RDBMS to Hadoop using Sqoop● Worked collaboratively to manage build-outs of large data clusters and real-time streaming with Spark.● Responsible for loading Data pipelines from web servers using Sqoop, Kafka, and Spark Streaming API● Monitoring the hive metastore and the cluster nodes with the help of Hue.● Developed Spark code using Scala and Spark-SQL/Streaming for faster processing of data. ● Created AWS EC2 instances and used JIT servers.● Developed various UDFs in Map-Reduce and Python for Pig and Hive.● Data Integrity checks have been handled using hive queries, Hadoop, and Spark.● Worked on performing transformations & actions on RDDs and Spark Streaming data with Scala.● Responsible for handling Streaming data from web server console logs.● Installed Oozie workflow engine to run multiple Hive and Pig Jobs.● Developed PIG Latin scripts for the analysis of semi-structured data.● Used Hive and created Hive tables and was involved in data loading and writing Hive UDFs.● Used Sqoop to import data into HDFS and Hive from other data systems.● Installed and configured Apache Hadoop to test the maintenance of log files in the Hadoop cluster.● Installed and configured Hive, Pig, Sqoop, Flume, and Oozie on the Hadoop cluster.● Worked on developing ETL processes (Data Stage Open Studio) to load data from multiple data sources to HDFS using FLUME and SQOOP and performed structural modifications using Map Reduce, and HIVE.● Involved in NoSQL database design, integration, and implementation● Loaded data into NoSQL database HBase.

● Suggestions on converting to Hadoop using MapReduce, Hive, Sqoop, Flume, and Pig Latin. ● Collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis● Extracted data from various locations and loaded them into the Oracle table using SQL*LOADER.● Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring, and troubleshooting, managing and reviewing data backups, and reviewing Hadoop log files.● Developed the Pig Latin code for loading, filtering, and storing the data.● Create, develop, modify, and maintain Database objects, PL/SQL packages, functions, stored procedures, triggers, views, and materialized views to extract data from different sources.● Handled Import of data from various data sources, performed transformations using Hive, and MapReduce, and loaded data into HDFS.● Extracted the data from Teradata into HDFS using Sqoop.● Analyzed the data by performing Hive queries and running Pig scripts to know user behavior.● Installed Oozie workflow engine to run multiple Hive. ● Developed Hive queries to process the data and generate the data cubes for visualization.● Handled importing of data from various data sources, performed transformations using Hive, and MapReduce, loaded data into HDFS, and Extracted the data from Oracle database into HDFS using Sqoop.● Importing and exporting data into HDFS and Hive using Sqoop.● Analyzed the data by performing Hive queries and running Pig scripts to know user behavior● Continuous monitoring and managing of the Hadoop cluster through Cloudera Manager.● Installed Oozie workflow engine to run multiple Hive and Pig jobs.● Developed Hive queries to process the data and generate the data cubes for visualization.● Worked on loading the data from MySQL to HBase where necessary using Sqoop