Integrated Azure HDInsight with Azure Data Lake Storage, ensuring seamless data flow and storage for analytics workloads, and enabling comprehensive data exploration and analysis. Utilized advanced data partitioning and indexing techniques in Azure Data Factory and Azure Synapse Analytics to improve query performance and reduce latency. Used Python to clean and pre-process raw data, ensuring data quality and consistency for analysis. Administered and managed the Hadoop ecosystem, including HDFS, Map Reduce, Hive, Pig and Spark, ensuring optimal performance and reliability.Implemented and maintained metadata catalogs for Spark SQL, ensuring data lineage and governance. Worked on Kafka topic partitioning strategies to optimize data distribution and parallel processing. Implemented data governance practices within Kafka and Scala, incorporating metadata management techniques to enhance data discoverability, lineage tracking, and overall data governance. Integrated Scala applications with monitoring and logging tools to proactively identify issues, analyze performance, and streamline troubleshooting processes Participated in the evaluation and implementation of Snowflake features and enhancements to improve data warehousing and analytics capabilities.Managed version control for Tableau workbooks and data sources, facilitating collaboration among team members and providing a clear audit trail for changes. Implemented global filters in Tableau to allow users to dynamically control multiple visualizations simultaneously, enhancing the overall user experience• Experienced in documentation of Power BI solutions, including data models, transformations, report specifications, to facilitate knowledge transfer and future maintenance. Maintained clear and comprehensive project documentation on GitHub for improved project understanding and on boarding. Used Azure DevOps and Jenkins pipelines to build and deploy different resources like Code and Infrastructure in Azure.

Designed and Developed ETL Processes in AWS Glue to migrate Campaign data from external sources like AWS S3, Text Files into AWS Redshift. Planned and executing data migration projects using Informatica, ensuring smooth transitions between systems while preserving data integrity and consistency.• Utilized Informatica's data profiling capabilities to analyze and understand the characteristics of data, identifying patterns, anomalies, and potential issues for proactive data management. Created interactive and informative data visualizations using Python libraries like Matplotlib, Seaborn, or Plotly to aid in data exploration and communication.Developed Spark applications in Scala for data cleansing, event enrichment, aggregation, de-normalization, and preparation for machine learning and reporting teams. Designed AWS architecture, Cloud migration, AWS EMR, Dynamo DB, Redshift and event processing using lambda function. Maintained data quality checks and validation processes on Snowflake, guaranteeing the accuracy and reliability of data throughout its lifecycle. Developed and tested disaster recovery plans for Snowflake, ensuring the availability and integrity of critical data in the event of system failures or unforeseen issues.Created visualization for decision support generated Dashboards with Quick filters, Parameters and preparing dashboards and worksheets in Tableau. Experienced with advanced Power BI topics such as complex calculations, table calculations, geographic mapping and performance optimization.• Utilized using version control systems to manage changes and track revisions in Power BI projects, ensuring traceability and collaboration in a team environment. Executed complex queries and utilized MongoDB's indexing strategies to enhance data retrieval performance, ensuring timely analysis and reporting.

Implemented Lakehouse architecture on Azure using Azure Data Lake, Delta Lake, Delta Tables and Azure Data bricks. Developed JSON scripts for deploying Pipelines in Azure Data Factory, facilitating data processing through SQL Activity. Implemented data governance policies and procedures, ensuring that data stored in Azure Blob Storage adheres to organizational standards and regulatory requirements.Implemented medium to large scale BI solutions on Azure using Azure Data Platform services (Azure Data Lake, Data Factory, Data Lake Analytics, Stream Analytics, SQL DW, Data bricks and NoSQL DB). Implemented proactive monitoring solutions in Informatica to track system performance, diagnose issues, and perform regular maintenance tasks, ensuring the reliability of data processes. Created interactive dashboards and reports using Python libraries like Plotly and Dash to provide real-time insights to business stakeholders.Developed Python scripts to interact with external APIs, facilitating seamless data exchange between different systems and enhancing the overall connectivity of the data ecosystem. Implemented Hadoop's security features, including authentication and authorization, to safeguard sensitive data and maintain compliance with industry regulations. Designed and implemented Spark applications for distributed data processing, utilizing RDDs and Data Frames. Developed Spark scripts using Scala Shell commands, ensuring efficient and customized data processing based on specific requirements.Conducted capacity planning for Tableau Server, optimizing resource utilization and ensuring optimal performance during peak usage periods. Established and optimized data connections in Tableau to various sources, ensuring accurate and up-to-date data retrieval. Integrated Power BI with data warehouse solutions, enabling seamless connectivity and ensuring the availability of up-to-date and accurate data for reporting.

Documented architecture, configurations, and procedures for AWS S3, Lambda, and AWS EMR implementations. Designed and developed Security Framework to provide fine grained access to objects in AWS S3 using AWS Lambda and Dynamo DB. Implemented complex data transformations using AWS Glue's built-in transformations and custom scripts, optimizing data for analytical processing. Used SQL to code program blocks, triggers, stored procedures, and functions. Importing and exporting flat files using Excel to SQL database.Created visually appealing and informative dashboards in Excel for presenting key performance indicators to stakeholders. Utilized Python libraries such as Pandas and NumPy for data manipulation and analysis, ensuring streamlined workflows. Conducted data cleansing and validation in Excel to ensure accuracy and consistency of information. Integrated Python scripts with relational databases (MySQL, PostgreSQL) for seamless data storage, retrieval, and maintenance, ensuring data integrity and accessibility. Maintained comprehensive documentation for MySQL databases, including schema designs, configurations, and troubleshooting procedures. Customized Jira workflows to align with project-specific requirements, enhancing team collaboration and efficiency. Implemented Jira reporting tools to generate project status reports, enhancing visibility for stakeholders. Created custom calculations and leveraging scripting languages within Tableau to perform advanced analytics and meet specific business requirements. Experienced in creating robust data models in QlikView and SAS, optimizing structures for efficient data retrieval and analysis, resulting in streamlined reporting processes. Optimized QlikView applications for faster data loading and improved user experience, enhancing overall system efficiency. Skilled in implementing and fine-tuning PostgreSQL full-text search functionality for information retrieval.