Dhanush Appala Email & Phone Number

Cary, North Carolina, United States

Designed and implemented scalable big data solutions using Azure Databricks, Azure Data Lake, and Azure Data Factory to process and analyze network traffic data.Developed real-time anomaly detection systems using Azure Stream Analytics and PySpark, enhancing Cisco’s ability to proactively address network issues.Engineered robust ETL pipelines in ADF and Databricks, handling terabytes of data daily, integrating external data sources, and automating data transformation.Integrated Apache Kafka as a fault-tolerant event streaming platform, enabling scalable ingestion of millions of events per minute.Developed and optimized data lakes and data warehouses using Azure SQL DW and Cassandra, improving data accessibility and analytics capabilities.Worked with DevOps to integrate machine learning models into production, automating deployments using Azure Functions and Azure Container Instances.Contributed to the improvement of CI/CD pipelines with Azure DevOps, streamlining development workflows and ensuring seamless code deployment.Provided technical mentorship to junior engineers and led cross-functional teams to enhance data management, pipeline performance, and security across cloud-based systems.ETL Developer with migrating legacy DTS packages to SSIS, optimizing data workflows for improved performance and scalability.Experienced in leveraging Palantir Foundry to drive advanced data integration, transformation, and analysis for large-scale network monitoring and machine learning initiatives. Proficient in building scalable pipelines and automating workflows within Palantir to enable real-time anomaly detection, streamline operational processes, and enhance data-driven decision-making. Skilled in designing customized dashboards and integrating diverse data sources into Palantir Foundry, ensuring seamless accessibility and actionable insights for proactive network performance management and operational efficiency

Dallas, Texas, United States

Developed scalable ETL pipelines in Python to automate the extraction, transformation, and loading of data from various sources into cloud-based data warehouses like Azure SQL Database and Google BigQuery.Utilized Azure Data Factory, Google Dataflow, and AWS Glue for large-scale data processing and integration, managing both structured and unstructured data efficiently.Used Terraform and Ansible to automate the provisioning and configuration of cloud-based data infrastructure, ensuring consistent and repeatable deployments across multiple environments.Integrated Jenkins and Azure DevOps to establish CI/CD pipelines for automated testing, deployment, and management of data applications, improving development workflow and deployment efficiency.Implemented performance tuning and optimization strategies for ETL pipelines to enhance data processing speed and reduce latency.Integrated ETL processes with data visualization tools like Power BI and Google Data Studio to deliver real-time insights and interactive dashboards for comprehensive data analysis.Configured and managed data storage solutions using Azure Data Lake Store Gen2, AWS S3, and Google Cloud Storage, ensuring secure and scalable storage of large data sets.Utilized monitoring tools such as Nagios, Splunk, and Azure Monitor to track the performance and health of ETL pipelines, including real-time logging and alerting for potential issues.Implemented security measures and managed access controls to protect sensitive data within cloud storage and data pipelines.Developed and maintained data integration solutions to seamlessly combine data from multiple sources, ensuring data consistency and integrity across the organization.Performed regular data quality assessments to ensure the accuracy, completeness, and reliability of the data processed through ETL pipelines.Coordinated with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions

Philadelphia, Pennsylvania, United States

Configured and automated cloud-based data infrastructure using Chef and Terraform, ensuring the seamless deployment of services across AWS EC2 instances and on-premises servers.Developed and managed ETL pipelines to automate the extraction of data from various sources, transforming it using Python scripts, and loading it into AWS S3 buckets, ensuring data consistency and availability for downstream processing.Managed and optimized data workflows by writing Chef Recipes/Cookbooks for the deployment and configuration of OS, applications, services, and packages, enhancing data processing efficiency.Developed Python scripts to automate the provisioning and scaling of cloud resources, including EC2, VPC, and other AWS services, ensuring high availability and fault tolerance of data pipelines.Utilized Terraform to automate the creation and management of AWS resources, including S3 buckets, IAM roles, and security groups, ensuring secure and scalable cloud storage solutions.Implemented CI/CD pipelines in Jenkins, integrated with GitHub, and used Ansible playbooks to automate the build, deployment, and configuration processes for data applications.Automated cloud monitoring and logging for data workflows using Nagios, Splunk, and AWS CloudWatch, ensuring the integrity and performance of data systems.Utilized Terraform and Chef to orchestrate and automate the deployment of ETL workflows across cloud-based environments, enabling scalable and efficient data integration and transformation processes.Defined and managed Ansible playbooks and roles to automate the continuous delivery process of data applications, enhancing the reliability and efficiency of data processing pipelines.Integrated log monitoring tools, including ELK Stack, for real-time analysis of data processing metrics and logs, facilitating proactive troubleshooting and optimization of data workflows.

India

Analyze, design, and build modern data solutions using Azure service to support data visualization. Understand the current Production state of the application and determine the impact of new implementation on existing business processes. Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks. Extract Transform and Load data from Source Systems to Azure Data Storage services using a combination of Azure Data Factory, Spark SQL, Azure Data Lake. Enhanced and optimized Spark/python jobs to aggregate, group, and run data mining tasks using the Spark framework. Worked on migration of data from On-prem SQL server to Cloud databases (Azure Synapse Analytics (DW) & Azure SQL DB). Designed Data Stage ETL jobs for extracting data from heterogeneous source systems, transforming and finally loading into a Data Warehouse. Experience in developing Spark applications using Spark-SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for Analyzing& transforming the data to uncover insights into customer usage patterns. Primarily involved in Data Migration using SQL, SQL Azure, Azure Storage, and Azure Data Factory. Responsible for estimating the cluster size, monitoring, and troubleshooting the Spark Data Bricks cluster.