Malathi Keerthi Email & Phone Number

Des Moines, Ia, Us

Des Moines, Iowa, United States

I have extensive experience in production support and migration tasks within Azure environments, managing Azure Data Factory pipelines, Azure Logic Apps, Azure Analysis Services, and Databricks scripts. I have successfully migrated SQL databases to Azure Data Lake, Azure SQL Database, and Data Bricks, ensuring seamless access control and efficient data integration. My role also involved designing impactful data visualizations using Matplotlib and optimizing Informatica ETL mappings through… Show more I have extensive experience in production support and migration tasks within Azure environments, managing Azure Data Factory pipelines, Azure Logic Apps, Azure Analysis Services, and Databricks scripts. I have successfully migrated SQL databases to Azure Data Lake, Azure SQL Database, and Data Bricks, ensuring seamless access control and efficient data integration. My role also involved designing impactful data visualizations using Matplotlib and optimizing Informatica ETL mappings through performance tuning and effective use of parameter files. Additionally, I implemented robust error handling in Talend jobs and utilized NumPy for advanced numerical computing tasks on large datasets.In the realm of big data, I have administered Hadoop Cluster environments, handling node commissioning, capacity planning, and performance tuning. I implemented Name-node HA services for high availability and developed data processing tasks using PySpark and Spark SQL for data ingestion and transformation. I worked with Kafka and Zookeeper for live data streaming and monitoring, and utilized NiFi for Spark streaming integration. My experience also includes working with Star and Snowflake schemas, developing reusable data pipelines for Snowflake, and optimizing ElasticSearch performance.I have a strong background in data visualization and reporting, having designed and built reports and dashboards using Tableau and Power BI, with customized security architecture. I integrated Git with CI/CD pipelines to automate testing and deployment, ensuring smooth delivery of data pipelines and applications. Additionally, I orchestrated automated testing and deployment workflows with GitHub and CI/CD platforms like Jenkins and Travis CI, facilitating rapid and reliable software delivery. Show less

Nyack, New York, United States

I have extensive experience creating run jobs on AWS Cloud, including extracting, transforming, and loading data into AWS Redshift using AWS Glue and S3 for data storage. I have utilized AWS Glue Catalog with a crawler to retrieve data from S3 and performed SQL query operations using AWS Athena. Additionally, I created various AWS Lambda functions and API Gateways to submit data, accessible via AWS Lambda functions, and fine-tuned ETL workflows using shell scripts and Informatica… Show more I have extensive experience creating run jobs on AWS Cloud, including extracting, transforming, and loading data into AWS Redshift using AWS Glue and S3 for data storage. I have utilized AWS Glue Catalog with a crawler to retrieve data from S3 and performed SQL query operations using AWS Athena. Additionally, I created various AWS Lambda functions and API Gateways to submit data, accessible via AWS Lambda functions, and fine-tuned ETL workflows using shell scripts and Informatica transformations, such as Source Qualifier, Joiner, Filter, and more.My expertise extends to designing and developing efficient data integration processes using Talend, optimizing Python code for performance, and implementing robust ETL pipelines using Python and frameworks like Apache Airflow. I have utilized Spark SQL API in PySpark to extract, load data, and perform SQL queries, improved Hadoop algorithms with Spark, and monitored and troubleshot Hadoop cluster performance. Furthermore, I have optimized Hive queries and integrated Flume with Kafka for real-time data ingestion in HDFS.In the realm of data visualization and reporting, I have developed interactive dashboards in Tableau, leveraging calculation fields, filters, and custom SQL queries to validate and reconcile data. I have created Power BI reports with various visualizations, including line charts, KPI, and scatter plots, and designed custom dashboards displaying MDM data insights using advanced Power BI techniques and DAX formulas. Additionally, I have utilized Git for version control, ensuring secure, tracked, and documented changes to code and data pipelines, and implemented security best practices in Git and GitHub repositories. Show less

Gurugram, Haryana, India

I have extensive experience working with Azure Data Factory, where I integrated data from on-prem SQL, Cassandra, and cloud sources such as Blob storage and Azure SQL DB, applying transformations and loading the data back to Azure Synapse. I developed data ingestion pipelines on Azure HDInsight Spark clusters using Azure Data Factory and Spark SQL, and created pipelines to load data from Azure Data Lake into staging SQLDB and subsequently into Azure SQL DB. Additionally, I implemented Change… Show more I have extensive experience working with Azure Data Factory, where I integrated data from on-prem SQL, Cassandra, and cloud sources such as Blob storage and Azure SQL DB, applying transformations and loading the data back to Azure Synapse. I developed data ingestion pipelines on Azure HDInsight Spark clusters using Azure Data Factory and Spark SQL, and created pipelines to load data from Azure Data Lake into staging SQLDB and subsequently into Azure SQL DB. Additionally, I implemented Change Data Capture (CDC) solutions in Informatica for real-time data integration, and designed configurable data delivery pipelines for scheduled updates using Python.In my role, I performed data cleaning, feature scaling, and feature engineering using Pandas and NumPy in Python. I developed Oozie scripts to manage and schedule Hadoop jobs, and created real-time dashboards on Hadoop using Platfora. I utilized Spark SQL API in PySpark to extract and load data, executed multiple Spark SQL queries for specific data gathering, and developed Spark Streaming programs to process near real-time data from Kafka. My experience also includes running queries using Impala, using Kafka producers for data ingestion, and developing PySpark scripts for encrypting raw data with hashing algorithms.Furthermore, I have a solid background in data visualization and BI tools. I worked on direct queries in Power BI to compare legacy and current data, generated reports, and created dashboards with action filters, parameters, and calculated sets. I conducted data blending and preparation using Alteryx and SQL for Tableau, designed interactive Tableau dashboards, and published data sources to Tableau Server. Additionally, I implemented GitLab CI/CD pipelines to automate builds, testing, and deployment processes, ensuring rapid and reliable software delivery. Show less

Hyderabad, Telangana, India

I have extensive experience in AWS and data engineering, including creating AWS Lambda functions and provisioning AWS EC2 instances. I implemented security groups and administered Amazon VPCs to ensure secure environments. Additionally, I developed Python serverless Lambda functions with concurrency and multithreading capabilities for efficient, asynchronous processing. My work also included implementing AWS CloudTrail to capture API call events within the AWS infrastructure.In the… Show more I have extensive experience in AWS and data engineering, including creating AWS Lambda functions and provisioning AWS EC2 instances. I implemented security groups and administered Amazon VPCs to ensure secure environments. Additionally, I developed Python serverless Lambda functions with concurrency and multithreading capabilities for efficient, asynchronous processing. My work also included implementing AWS CloudTrail to capture API call events within the AWS infrastructure.In the realm of big data, I worked on Kafka REST API to collect and load data into the Hadoop File System, using Sqoop for relational database integration. I developed various Python scripts to detect vulnerabilities through SQL injection, permission checks, and performance analysis, and created scripts for data migration from proprietary databases. My expertise extends to real-time data processing with Kafka and Spark Streaming, converting data into RDDs, and storing it in HDFS as Parquet format. I also implemented Apache Drill on Hadoop to integrate SQL database data, used Spark-Streaming APIs for data transformations, and developed custom RDDs in Scala for data transformations.My proficiency in data analytics includes using Spark API over Hadoop YARN for data analytics with Hive and developing a high-availability ETL framework for Hadoop using Spark. I conducted Tableau data source optimizations with Tableau Prep Builder for data cleaning and integration. In Power BI, I created DAX calculations and measures for advanced analytics, integrated Power BI with various data sources for live connections, and set up automated data refresh schedules to ensure data accuracy and timeliness. Show less