• Proficient working with Azure data bricks, PySpark, HDInsight, u-sql, t-sql, spark, sql, azure ADW and hive to load and transform data. Planned, deployed, and maintained of Data Cloud solutions using Microsoft Azure. Used Data Factory to Custom Process design of Transformation and build the data pipeline. Working with the development, Infrastructure and QA teams to maintain the quality of deployment. Processed more than 30 Tb of data and dumped it to SQL server by setting up Spark Cluster and built different Spark jobs to run data actions and Transformations. Loaded Cookie level data and real-time streaming by setting up Spark cluster and Hadoop and integrated with different ecosystems like Hive, HDFS/Blob Storage/Data Lake

• Designed and implemented robust data pipelines using Apache Spark and Kafka to process and stream transaction data in real-time. This involved integrating Apache Kafka for efficient data streaming and processing, ensuring low latency and high availability. Utilized Hadoop Distributed File System (HDFS) for scalable storage of both raw and processed transaction data, ensuring fault tolerance and scalability to handle large volumes of data. Implemented data transformation and validation processes using Python and SQL, ensuring data integrity and consistency across various data sources. Developed basic data visualizations using Tableau to monitor transaction trends and patterns, providing actionable insights to stakeholders. Automated cloud deployments using chef, python, and AWS Cloud Formation Templates. Used SQL tools like Teradata SQL Assistant to run SQL queries and validate the data in the warehouse and marts. Migrated existing self-service reports and ad hoc reports to Power BI.

• Spearheaded the optimization of database performance, resulting in a 30% increase in query efficiency and a significant reduction in report generation time. Achieved a 40% increase in report effectiveness through customized report features, including Filters, Calculations, Prompts, Summaries, and Functions. Developed highly efficient stored procedures, functions, and database views, leading to a 40% improvement in data retrieval speed and enhanced overall system responsiveness. Advised on the optimization of slow performance queries, implementing strategic tuning measures that resulted in a 25% enhancement in database query execution time. Transformed business requirements into actionable BI solutions, driving a 20% increase in data-driven decision-making across departments. Integrated Flask-Application into OKTA. Engineered robust SSIS packages for seamless data integration from diverse sources, ensuring a 50% reduction in data loading time and enhancing data accuracy by 25%. Implemented advanced data scrubbing techniques, including validation checks during staging, resulting in a 30% improvement in data quality and integrity. Worked on ETL Migration services by developing and deploying AWS Lambda functions for generating a serverless data pipeline which can be loaded to Glue Catalog and can be queried from AWS Athena.

• Created Business Requirement Document, Functional Requirement Document, use cases, test scenarios, test cases for testing the functional requirement. Performed Data Blending techniques for deep insight of data. Created ad-hoc reports to users in Tableau by connecting various data sources. Implemented and maintained SQL Script, involved in making code changes to the scripts and revalidating them. Design analytical solutions for reporting purposes and work on various ad-hoc data pulls in different formats like excel, pivot tables, and flat files. Facilitated JAD sessions to collect requirements from businesses and prepared business requirements that provided appropriate scope of work for report development. Involved in gathering, documenting, and verifying business requirements. Assisted Design Team in preparing SRS, Software Design Document (SDD), User Interface Design, Application Architecture & Database Modeling. Worked on Data mapping, logical data modeling used SQL queries to filter data within the database tables. Developed data reconciliation SQL queries and reports to validate overall data migration and support all scheduled ETL jobs for batch processing. Created reports and user stories using Tableau and Power BI.

• Propose new quantitative analysis on various projects on customer retention analysis, promotion program evaluation and anomaly detection, co-operate with client managers and prepare the slides. Research and test statistical models and machine learning to analyze and predict customer’s behaviors and visualize the results. Proficient in creating calculated fields, parameters, and groups to generate custom views and analysis, sharing reports and dashboards with colleagues and clients, including publishing to Tableau Server or Tableau Online. Performed data cleaning, exploration and analysis using Python. Feature Engineering through Data Transformation and Data Manipulation in Python. Performed tests on import programs, analyzed results and modified programs. Worked on model building using Machine Learning algorithms like Linear Regression, Logistic Regression, SVM, KNN, and Decision Trees. Designing automated dashboards and data visualization to extract relevant information. Sorting, printing, and summarizing the data and modifying and combining datasets by different packages. Deployed SSIS packages and reports in Dev and QA environments and created deployment documentation.