Rohith A Email & Phone Number

New Jersey, United States

• Led the design and automation of complex ETL pipelines on Snowflake DB, using Apache Spark and Python APIs like PySpark for smooth data transformation and integration from various sources.• Developed a streamlined data mart adhering to the Star schema for efficient analytics.• Managed and optimized Elastic pool databases, scheduling T-SQL procedures through Elastic jobs to ensure effective data processing.• Demonstrated proficiency across a range of Azure cloud services including HDInsight, Data Lake, Databricks, Blob Storage, Data Factory, Synapse, SQL, SQL DB, DWH, and Data Storage Explorer, delivering robust data engineering solutions.• Oversaw end-to-end data ingestion into Azure Services such as Azure Data Lake, Azure Storage, Azure SQL, and Azure DW, facilitating seamless data processing and analysis within Azure Databricks.• Implemented advanced Airflow orchestration techniques to simplify data migration from Hive external tables to Azure blob storage, optimizing Hive jobs through innovative partitioning and bucketing strategies.• Developed advanced Spark applications in Azure Databricks using Spark-SQL for comprehensive data analysis, uncovering valuable insights into customer behavior and usage patterns.• Demonstrated expertise in performing ETL operations within Azure Databricks, establishing JDBC connectors to connect with various relational database source systems.• Engineered robust and scalable ETL pipelines from Azure Data Lake to Data Warehouse, addressing complex Medicaid and Medicare data requirements.

Texas, United States

• Developed Spark applications using Scala and implemented Apache Spark data processing projects to manage data from various RDBMS and streaming sources.• Enhanced the performance and efficiency of existing Spark jobs and transformed MapReduce scripts into Spark SQL.• Scheduled Spark/Scala jobs using Oozie workflow in Hadoop Cluster and created detailed design documentation for source-to-target transformations.• Created Python code for different tasks, dependencies, and time sensors for each job to manage workflows and automation using the Airflow tool.• Designed and deployed a scalable data warehousing solution on AWS using Snowflake, facilitating storage and processing of large volumes of structured and semi-structured data.• Participated in designing and deploying multi-tier applications using various AWS services such as EC2, Route53, S3, RDS, DynamoDB, SNS, SQS, and IAM, focusing on high availability, fault tolerance, and auto-scaling using AWS CloudFormation.• Developed AWS Data pipelines from various data resources in AWS, including AWS API Gateway to receive responses from AWS Lambda, retrieving and converting data responses into JSON format, and storing them in AWS Redshift.• Established and maintained the Hadoop cluster on AWS EMR, utilizing AWS services like EC2 and S3 for processing and storage of small datasets.• Utilized Spark Streaming to collect data from an AWS S3 bucket in real-time, performing necessary transformations, aggregations, and persisting the data in HDFS.• Developed scalable AWS Lambda code in Python for nested JSON files, performing tasks such as conversion, comparison, and sorting.• Implemented AWS Glue catalog with a crawler to retrieve data from S3 and execute SQL query operations.

Texas, United States

• Utilized Informatica Client tools including Power Center Designer, Workflow Manager, Workflow Monitor, and Repository Manager extensively for data integration and ETL processes.• Employed Kafka for live streaming data and conducted analytics. Leveraged Sqoop for transferring data between relational databases and Hadoop.• Configured real-time data pipelines using Kafka for streaming data ingestion and Spark Streaming for real-time consumption and processing.• Loaded data from Web servers and Teradata using Sqoop, Flume, and Spark Streaming API.• Proficient in writing optimized SQL queries in Teradata for extracting data from diverse source systems such as relational databases, flat files, APIs, and web services.• Developed MapReduce/Spark Python modules for machine learning and predictive analytics in Hadoop on AWS. Implemented Python-based distributed random forest via Python streaming.• Wrote multiple MapReduce programs for data extraction, transformation, and aggregation from various file formats including XML, JSON, CSV, and other compressed file formats.• Extracted data from heterogeneous sources like Oracle and Flat Files.• Developed complex mappings using the Informatica Power Center tool.• Performed data extraction from Oracle, Flat files, Excel files, and executed complex transformations such as joiner, expression, aggregate, lookup, stored procedure, filter, router transformation, and update strategy transformations to load data into target systems.• Collaborated with Data modelers to develop STAR schemas.• Analyzed the existence of source feed in the existing CSDR database.• Managed high volumes of Informatica workflow migrations on a daily basis.• Reviewed Informatica ETL design documents and ensured adherence to correct standards by working closely with development teams

New Jersey, United States

Proficient in exploratory data analysis using Python libraries such as NumPy, Matplotlib, and Pandas.Skilled in quantitative analysis and data mining, adept at uncovering trends and insights beyond the numbers.Developed complex SQL queries and scripts for data extraction and aggregation, ensuring data accuracy and meeting business requirements.Prepared high-level analysis reports using Excel and Tableau, identifying billing patterns and outliers.Extensive experience with Tableau sorting and filters, including basic, quick, context, and condition filters.Identified and documented data quality limitations, performed data validation with standard SQL queries, and created Excel summary reports.Gathered analytical data to develop functional requirements using data modeling and ETL tools.

Master'S Degree, Information Technology

Bachelor Of Commerce - Bcom, Computer Engineering, A