Gowtham C Email & Phone Number

▪ Developed ETL pipelines in and out of the data warehouse, creating major regulatory and financial reports using advanced SQL queries in Azure Synapse Analytics.▪ Involved in creating ETL pipelines to ingest data into Azure datalake.▪ Automate the pipeline using Airflow by creating dependencies between each job and schedule them on the basis of daily/weekly/monthly .▪ Developed Spark core and Spark SQL scripts using Python for faster data processing. Transformed the data using Spark applications for analytics consumption.▪ Worked on creating incremental spark jobs to move data from azure to snowflake.▪ Used Kafka connector for reading CDC streaming feed from Cosmos Db, mongo DB in real time into Kafka, transform and write into Azure data lake▪ Experience in Big data Eco-System in Azure using Spark, Kubernetes, Airflow, Databricks.▪ Leveraged Azure DevOps for continuous integration and continuous deployment (CI/CD) pipelines, ensuring rapid and reliable software delivery.▪ Supported the production environment and debug issues using Azure Monitor logs, Grafana▪ Written ETL jobs to read from web APIs using REST and HTTP calls, loading into Azure Data Lake Storage using pyspark.▪ Designed and implemented a hybrid cloud solution using Azure, integrating on-premises infrastructure with Azure services for high availability and disaster recovery capabilities.▪ Implemented Change Data Capture technology in Azure Data Factory for loading deltas to the Data Warehouse.▪ Integrated diverse data sources (databases, cloud storage, APIs) into PowerBI, enabling seamless data connectivity for visualization.▪ Contributed to consulting on Azure Synapse Analytics Solution Architecture, Development, and deployment to foster a data-driven culture.▪ Utilized Git for version control and collaborated with a cross-functional team, ensuring smooth integration and code management.▪ Implemented Terraform scripts for IaC to automate and manage cloud infrastructure deployments.

Experience in Building Enterprise Data Lake in AWS Cloud. Designing and Developing ETL framework in Hadoop using Sqoop, Oozie, Distcp, HDFS.Building components to extract and load data from any RDBMS source like Netezza, Redshift, Oracle, Mysql toHadoop File system.Designing and Building Materialistic views in Hive as S3data storage and metadata in Metastore.Building components to load incremental data into materialistic views whenever there is change in baseobjects.Building components to replicate and sync Metastorebetween two or more clusters.Storing and retrieving data in Encrypted S3 and RDS in AWS.Building data validation and comparison module between any RDBMS systems vs. HDFS using Apache Spark.Applying business logic rules in-memory processing using AWS EMR Spark. Working with multiple security data layerlike Kerberos for Authentication for data stored in cluster and Sentry for Authorization of data stored in HDFS or S3.Structured PySpark scripts generated by AWS Glue for performing serverless ETL.Migrating ETL framework build in Hadoop Cloudera to AWSElastic Map Reduce Service.Running AWS EMR in Spot instances and scaling up/downcore nodes based on demand.Building Transient EMR cluster using AWS lambda and AWSCloud Formation Stack.Developed applications and designed processes for transformation and data management from company-wide-databases.Created Spark JDBC APIs for importing/exporting data from Snowflake to S3 and vice versa.Experienced in working with EMR cluster and S3 in AWS cloud, Aurora.Automated the data pipeline to ETL all the Datasets along with full loads and incremental loads of data.

• Analyze health management programs including data collection, validation and outcomemeasurement which include:o Financial, pharmacy, claims, provider, and member datao IRS, CMS, HHSC, HEDIS reportingo Internal data cleansing and data reconciliation analysiso Trend analysis in various functional areas of health care management. Automated the process of generating reports and Interactive dashboards through MicroStrategy, Teradata, MS-Excel, MS-Access, and SQL using Business Objects interface and direct links to core databases (ODS/EDW).• Automated the dashboard creation in MicroStrategy using Microsoft SSIS, Teradata, visualstudio and embedded them into SharePoint.▪ Designed and implemented a real-time data processing pipeline using Azure Event Hubs and Azure Databricks, enabling the ingestion, transformation, and analysis of streaming data for immediate insights and decision-making.▪ Orchestrated the migration of on-premises big data infrastructure to Azure, leveraging services like Azure HDInsight, Azure Data Lake Storage, and Azure Synapse Analytics for scalable and cost-effective data processing solutions.▪ Developed an end-to-end data pipeline for a retail analytics platform, integrating data from various sources and generating visualizations using Power BI for actionable business insights.▪ Implemented a data lake architecture using Azure Data Lake Storage and Azure Synapse Analytics, enabling the organization to store and analyze diverse data types from different sources in a centralized and scalable repository for advanced analytics and reporting.▪ Worked with building data warehouse structures, and creating facts, dimensions, aggregate tables, by dimensional modeling, Star and Snowflake schemas.▪ Applied transformation on the data loaded into Spark Data Frames and done in memory data computation to generate the output response.▪ Used Infoworks ETL tool to migrate the data from Teradata, Oracle Db and other sources to Google Big query.

Hyderabad, Telangana, India

▪ Analyze the Hadoop Bigdata distributed computing requirements from the system users and then design and develop the Map reduce and spark components of the big data system to meet the user requirements. ▪ Designed and implemented distributed data processing pipelines using Apache Spark, Hive, Scala, Python, Airflow DAGs other tools and languages in Hadoop ecosystem.▪ Created HBase tables from Hive and Wrote HiveQL statements to access HBase table's data.▪ Developed complex Hive Scripts for processing the data and created dynamic partitions and bucketing in hive to improve the query performance.▪ Developed MapReduce applications using Hadoop Map-Reduce programming framework for processing and used compression techniques to optimize MapReduce Jobs.▪ Designed and lead the SAS to Pyspark migration project and played a crucial role in successful implementation of the end-to-end migration process.▪ Designed and implemented scalable cloud data and analytical solutions for various public and private cloud platforms using Azure. Involved in building Data Warehouse in Azure platform using Azure data bricks and data factory.▪ Providing system analysis, design, development and implementation of data analytics (ETL) solutionsTranslated technical specifications, and/or design models into code for new or enhancement projectsTroubleshooting software and processes for data consistency and integrity.▪ Resolve Spark and Yarn resource management issues in Spark including Shuffle issues, Out of Memory issues, heap space errors and schema compatibility.▪ Monitor and troubleshoot performance of applications and take corrective actions in-case of failures and evaluate possible enhancements to meet SLAs.▪ Translates business specifications into design specifications and code. Responsible for writing complex programs, ad hoc queries, and reports.▪ Ensure that all code is well structured, includes sufficient documentation, and is easy to maintain and reuse.

Master'S Degree, Information Technology, 3.9

Bachelor'S Degree, Information Technology, 3.7