Ruthvik K Email & Phone Number

United States

Green Bay, Wisconsin, United States

Analyzed, designed, and constructed modern data solutions using Azure PaaS services to facilitate data visualization.Orchestrated the Extract, Transform, and Load (ETL) process from diverse source systems to Azure Data Storage services using a combination of Azure Data Factory, Spark SQL, Python, and Azure Data Lake Analytics.Managed the ingestion of data into multiple Azure services such as Azure Data Lake, Azure Storage, Azure SQL, and Azure Data Warehouse, processing it within Azure Databricks.Developed robust pipelines within Azure Data Factory to extract, transform, and load data from various sources, including Azure SQL, Blob storage, Azure SQL Data Warehouse, and write-back tools.Built data pipelines using Azure Data Factory and Azure Databricks, loading data to Azure Data Lake.Developed jobs/scripts using ETL tools to translate client requirements into application software, enabling data loads across various platforms.Participated in the migration of legacy applications to the Google Cloud Platform (GCP), managing services like Compute Engine, Cloud Storage, BigQuery, VPC, Stack Driver, Load Balancing, and IAM.Architected and implemented ETL and data movement solutions using Azure Data Factory and SSIS.Converted Hive/SQL queries into Spark transformations using Spark RDDs and Scala.Created a Spark application for loading CSV file data and applying business validation, storing valid data in Hive partition tables and invalid data in error tables.Developed ETL pipelines using a combination of Python and Snowflake.Developed and tested Python-based web applications interacting with MySQL.Used the FLATTEN table function to produce a lateral view of VARIANT, OBJECT, and ARRAY columns.Developed PySpark and Spark-SQL applications to extract, transform, and aggregate data from diverse file formats, uncovering insights into customer usage patterns.

Jacksonville, Florida, United States

Architected data pipeline frameworks for efficient ingestion and ETL workflows, ensuring optimal dataprocessing. Developed pipelines for syllabus and curriculum data, transferring data from web services to HBASEand Hive tables.  Contributed to migrating objects from on-premises systems to Snowflake, including establishing snowpipes, data sharing, building table structures, schemas, and managing Snowflake clones. Facilitated datatransfer from Teradata to Snowflake for consumption on Databricks. Deployed AWS Data Pipeline to orchestrate an Amazon EMR cluster for cleaning and processing webserver logs stored in an Amazon S3 bucket. Managed AWS EMR clusters and access using AmazonIAM. Developed data pipelines for real-time use cases using Kafka, Flume, and Spark, streaming real-timedata with Kafka and Spark streaming, and storing processed data in Cassandra. Established a Data Lake, loaded and transformed structured and semi-structured data, and designed data pipelines based on business requirements. Used Airflow and Oozie Coordinators for scheduling andautomation. Conducted data analysis, feature selection, and feature extraction using Apache Spark to deriveactionable insights. Utilized Apache Kafka for consolidating web log data from multiple servers, enabling downstreamanalysis. Configured Spark streaming with Kafka Streams for retrieving and storing information inHDFS. Conducted bulk loading of JSON data from an S3 bucket into Snowflake, utilizing Snowflake functionsfor parsing semi-structured data with SQL. Configured multiple AWS services like EMR and EC2 to maintain compliance with organizationalstandards. Developed code for extracting data from Oracle Database and loading it into the AWS platform using AWS Data Pipeline.Implemented a full-text search feature with multi-tenancy elastic search after collecting real-time datathrough Spark streaming.

Schaumburg, Illinois, United States

Core-Site, HDFS-Site, YARN-Site, and Map Red-Site. For widely adopted Hadoop distributions likeHortonworks, Cloudera, or MapR, configuration files are set up automatically during startup, reducingthe need for manual intervention by Hadoop administrators.  Utilized Sqoop for importing data from relational databases such as MySQL and Oracle.  Engaged in importing both structured and unstructured data into HDFS.  Responsible for fetching real-time data through Kafka and processing it using Spark and Scala.  Leveraged Kafka for importing real-time weblogs and ingested the data into Spark Streaming.  Developed business logic using Kafka Direct Stream within Spark Streaming, implementing necessarybusiness transformations. Created and executed a real-time streaming ETL pipeline using Kafka Streams API.  Utilized Hive for web interfacing and storing data within Hive tables.  Migrated MapReduce programs to Spark transformations using Spark and Scala.  Familiar with Spark Context, Spark-SQL, and Spark YARN.  Managed data extraction and ingestion from various sources into the Hadoop Data Lake, constructingETL pipelines via Sqoop, Hive, and Spark. Designed and developed PySpark applications using Python, evaluating Spark's performance againstHive.  Created Spark modules, intricate queries, functions, and views to generate reports for business users.  Generated automated Python scripts to transform data from various sources and create ETL pipelines.  Developed comprehensive end-to-end data processing pipelines, including data reception via distributed messaging systems like Kafka, persisting data into Hive, and enhancing data with PySpark. Implemented strategies like Sequence files, RC files, Map side joins, bucketing, and partitioning inHive to enhance performance and storage efficiency.  Developed Enterprise JavaBeans (EJBs) to handle business logic processing.  Designed and constructed user interfaces for an ERP project using Java Server Faces (JSF).