Meghana S Email & Phone Number

Florida, United States

• Managed end-to-end Big Data flow within the application, including data ingestion from upstream sources to HDFS, as well as processing and analysis of data in HDFS.Worked extensively with PySpark on EMR and Databricks, leveraging the Hadoop ecosystem for efficient data processing.Utilized Spark core, Spark SQL, and Spark Streaming for real-time data processing and analysis.Designed and implemented a learner data model that receives real-time data from Kafka and persists it in Cassandra, enabling real-time data storage and analysis.Utilized various AWS cloud components, such as EMR, S3, EC2, Athena, RDS, Dynamo DB, Lambda, IAM and Glue

Bloomington, Illinois, United States

•Designed and developed proof of concepts (POCs) in Spark using Scala to compare the performance of Spark with MapReduce and Hive.Demonstrated proficiency in Azure Cloud Services, including Azure Synapse Analytics, SQL Azure, Data Factory, Azure Analysis Services, Application Insights, Azure Monitoring, Key Vault, and Azure Data Lake.Created batch and streaming pipelines in Azure Data Factory (ADF) using Linked Services, Datasets, and Pipeline components to extract, transform, and load data.Implemented Azure Data Factory (ADF) batch pipelines to ingest data from relational sources into Azure Data Lake Storage (ADLS gen2) in an incremental fashion, followed by data cleansing and loading into Delta tables.Developed Azure Logic Apps to trigger data ingestion when a new email with an attachment is received and load the file to blob storage.Implemented CI/CD pipelines using Azure DevOps in the cloud, leveraging tools such as GIT, Maven, and Jenkins plugins.

Indianapolis, Indiana, United States

•Worked on development of data ingestion pipelines using ETL tool, Informatica & bash scripting with big data technologies including but not limited to Hadoop, Hive, Spark, Kafka.Experience in developing scalable & secure data pipelines for large datasets.Implemented data streaming capability using Kafka and informatica for multiple data sources.

Hyderabad, Telangana, India

• Provided technical assistance and training in using tools such as Jira and Confluence tools and worked with querying RDBMS such as Oracle, MYSQL and SQL Server by using PL/SQL for data integrity.• Created Multibranch Pipeline in Jenkins to implement different Jenkins files for different branches in the same application thus it automatically discovers, manages, and executes Pipelines for branches which contain a Jenkins file in it.• Administered Linux servers for several functions including managing Apache/Tomcat server, mail server, MySQL database, and firewalls in both development and production.• Automated the Jenkins build using web hooks and SCM polling to trigger branch specific builds promptly when new changes are pushed to code repository.• Installed Jenkins on a Linux machine and created a master and slave configuration through a build farm for implementation of multiple parallel builds.• Analyzed memory, thread, CPU on Dynatrace for debugging run time issues on applications running on cluster.• Managed Linux staging and testing environments and automated application packaging and deployments. • Setup Continuous integration with Jenkins and make use of a wide range of plugins available to set up smooth developer-friendly workflows.

Bachelor'S Degree

Master'S Degree, Computer Science