Rohith V Email & Phone Number

Dania, Florida, United States

• Designed and implemented various layers of data lake architecture, including star schema designs in BigQuery. Leveraged Google Cloud Functions with Python to seamlessly load data into BigQuery upon the arrival of CSV files in GCS buckets. • Utilized Cloud Dataflow with Python to efficiently process and load both bound and unbound data from Google Pub/Sub topics into BigQuery. • Proficient in building multiple data pipelines, orchestrating end-to-end ETL and ELT processes for data ingestion and transformation within GCP, while effectively coordinating tasks among team members. • Played a key role in migrating on-premises Hadoop systems to Google Cloud Platform (GCP), ensuring smooth transition and optimization of resources. • Successfully migrated previously written cron jobs to Airflow/Composer within GCP infrastructure, enhancing scheduling and workflow management capabilities. • Provided support for existing GCP data management implementations, ensuring smooth operation and troubleshooting when necessary. • Developed GCP BigQuery authorized views to enforce row-level security and facilitate data sharing among different teams. • Extensive experience in IT data analytics projects, with hands-on involvement in migrating on-premise ETL processes to GCP using native tools such as BigQuery, Cloud Dataproc, Google Cloud Storage, and Composer. • Designed pipelines incorporating Apache Beam, KubeFlow, and Dataflow, effectively orchestrating jobs within the GCP environment. • Successfully demonstrated proof-of-concepts (POCs) for migrating on-premises workloads to Google Cloud Platform using GCS, BigQuery, Cloud SQL, and Cloud Dataproc.

Weehawken, New Jersey, United States

• Played a pivotal role in designing and deploying multi-tier applications utilizing a comprehensive array of AWS services such as EC2, Route53, S3, RDS, DynamoDB, SNS, SQS, and IAM within AWS CloudFormation. Emphasized on achieving high-availability, fault tolerance, and auto-scaling capabilities.• Spearheaded the development of AWS Data pipelines, seamlessly integrating various data resources within AWS, including AWS API Gateway to interface with AWS Lambda, retrieving and converting data responses into JSON format, and storing them in AWS Redshift.• Engineered scalable AWS Lambda functions in Python, specializing in handling nested JSON files, encompassing tasks such as conversion, comparison, sorting, and more.• Developed Spark Applications utilizing Scala, implementing Apache Spark data processing projects to manage data from diverse RDBMS and Streaming sources, ensuring efficient handling of large-scale data processing tasks.• Successfully optimized the performance and efficiency of existing Spark jobs, transitioning MapReduce scripts to Spark SQL for enhanced data processing capabilities.• Proficient in real-time data collection from AWS S3 buckets using Spark Streaming, executing necessary transformations and aggregations, and persisting the processed data in HDFS.• Implemented AWS Glue catalog with crawler functionality to extract data from S3 and execute SQL query operations for streamlined data processing workflows.• Designed and implemented robust and scalable data integration pipelines, facilitating seamless transfer of data from S3 buckets to RedShift databases using Python and AWS Glue.• Established and maintained Hadoop clusters on AWS EMR, utilizing AWS services like EC2 and S3 for processing and storing small datasets efficiently.• Developed Python scripts to manage various tasks, dependencies, and time sensors for each job, ensuring smooth workflow management and automation using the Airflow tool.

Hyderabad, Telangana, India

• Extensively engaged with the Azure cloud platform, including HDInsight, Data Lake, Databricks, Blob Storage, Data Factory, Synapse, SQL, SQL DB, DWH, and Data Storage Explorer.• Orchestrated the ingestion of data into one or more Azure services such as Azure Data Lake, Azure Storage, Azure SQL, and Azure DW, followed by data processing within Azure Databricks.• Designed and configured Azure Cloud relational servers and databases, meticulously analyzing current and future business requirements.• Engineered data ingestion pipelines on Azure HDInsight Spark cluster using Azure Data Factory and Spark SQL, ensuring seamless data flow.• Configured Input & Output bindings of Azure Functions with Azure Cosmos DB collection to facilitate seamless data interaction.• Developed robust ETL pipelines in Azure Data Factory (ADF), utilizing Linked Services from various sources to load data into Azure SQL Data Warehouse.• Implemented Elastic pool databases and scheduled Elastic jobs to execute T-SQL procedures, ensuring efficient data management.• Developed Spark applications in Azure Databricks utilizing Spark-SQL for comprehensive data extraction, transformation, and aggregation, aimed at uncovering insights into customer usage patterns.• Proficiently executed ETL operations in Azure Databricks by establishing connections to different relational database source systems using JDBC connectors.• Conducted data migration from Azure Blob storage to Azure Data Lake leveraging Azure Data Factory (ADF).• Engineered robust and scalable ETL pipelines from Azure Data Lake to Data warehouse applications for Medicaid and Medicare data using Azure Databricks.• Automated data engineering ETL pipeline over Snowflake DB using Apache Spark, integrating data from disparate sources with Python APIs like PySpark, and consolidating them in a data mart (Star schema).

Hyderabad, Telangana, India

• Extracted diverse data sets from HDFS, encompassing customer behavior, sales and revenue data, as well as supply chain and logistics data.• Utilized Apache Nifi, an open-source data integration tool, to seamlessly transfer the extracted data to AWS S3, ensuring robust and scalable dataflows.• Employed Python scripts to meticulously validate and cleanse the data before storing it in S3, ensuring data integrity and quality.• Leveraged PySpark, a distributed computing framework with Python API, to efficiently process and transform the data, catering to the requirements of big data processing.• Loaded the transformed data into AWS RedShift data warehousing for comprehensive data analysis and insights generation.• Scheduled the data pipeline using Apache Oozie, a workflow scheduler system tailored to managing Apache Hadoop jobs, ensuring streamlined data processing operations.• Spearheaded the development and maintenance of a library comprising custom Airflow DAG templates and operators, enhancing consistency and code quality across the team.• Led a team of three data engineers in the design and implementation of a sophisticated data ingestion and processing pipeline for a new data source, resulting in a 50% reduction in time to insights.• Conducted in-depth data analysis in HDFS using Apache Hive, a robust data warehouse software facilitating seamless querying and management of large datasets.• Transformed Hive queries into PySpark transformations utilizing PySpark RDDs and Data Frame API, optimizing data processing operations.• Monitored the performance of the data pipeline and applications using Grafana, ensuring efficient and reliable data flow.• Utilized GitHub as a version control system to manage code changes, facilitating collaboration and code management within the team.• Developed insightful visualizations and dashboards using Tableau for reporting and business intelligence purposes, providing stakeholders with actionable insights.

Hyderabad, Telangana, India

• Proficient in diverse projects encompassing machine learning, big data analysis, data visualization, R and Python development, Unix, and SQL.• Conducted exploratory data analysis leveraging tools such as NumPy, matplotlib, and pandas, ensuring comprehensive data understanding.• Possesses expertise in quantitative analysis, data mining, and effectively presenting insights from data, transcending mere numbers to uncover trends and actionable insights.• Extensive experience analyzing data utilizing Python libraries including Pandas, NumPy, SciPy, and Matplotlib, ensuring robust data exploration and analysis.• Created complex SQL queries and scripts to extract, aggregate, and validate data accuracy, while also excelling in business requirements gathering and translating them into clear and concise specifications.• Prepared high-level analysis reports using Excel and Tableau, providing valuable insights including identification of billing patterns and outliers.• Proficient in Tableau functionalities such as Basic Sorting, Basic Filters, Quick Filters, Context Filters, Condition Filters, Top Filters, and Filter Operations, ensuring efficient data visualization and analysis.• Identified and documented limitations in data quality, wrote standard SQL queries for data validation, and developed analytical data to derive functional requirements using data modeling and ETL tools.• Demonstrated versatility in reading data from various sources including CSV files, Excel, HTML pages, and SQL, performing comprehensive data analysis, and writing to diverse data destinations.• Leveraged Lambda functions such as filter(), map(), and reduce() with Pandas DataFrame, executing various data operations efficiently.• Utilized Pandas API for analyzing time series data and developed regression test frameworks for new code implementations.• Developed and managed business logic through backend Python code, ensuring robust and efficient data processing.