•Performed data investigation to discover correlations trends and the ability to explain them. Worked with Data Engineers, Data Architects, to define back-end requirements for data products (aggregations, materialized views, tables – visualization) •Performing data analysis, statistical analysis, generated reports, listings, and graphs using SAS tools, SAS/Graph, SAS/SQL, SAS/Connect and SAS/Access. •Developing Spark applications using Scala and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats. Using Kafka and integrating with the Spark Streaming. •Migrate data from on-premises to AWS storage buckets. Developed data analysis tools using SQL and Python code. Developed various Mappings with the collection of all Sources, Targets, and Transformations using Informatica Designer. •Designed and implemented Sqoop for the incremental job to read data from DB2 and load to hive tables and connected to Tableau for generating interactive reports using Hive server2. •Used Spark Streaming to receive real time data from the Kafka and store the stream data to HDFS using Python and NoSQL databases such as HBase and Cassandra. •Collected data using Spark Streaming from AWS S3 bucket in near-real-time and performs necessary Transformations and Aggregation on the fly to build the common learner data model and persists the data in HDFS. Used Apache NiFi to copy data from local file system to HDP. •Worked on Dimensional and Relational Data Modeling using Star and Snowflake Schemas, OLTP/OLAP system, Conceptual, Logical and Physical data modeling using Erwin. •Automated the data processing with Oozie to automate data loading into the Hadoop Distributed File System. •Experience in Converting existing AWS Infrastructure to Server less architecture (AWS Lambda, Kinesis), deploying via Terraform and AWS Cloud Formation templates. •Architect and design server less application CI/CD by using AWS Server less (Lambda) application model.

•Worked with Azure BLOB and Data Lake storage and loading data into Azure SQL Synapse analytics.•Worked on Azure Databricks for mounting, updating mounting blob storages, creating python, SQL, Scala notebooks, and secret scopes backed by Databricks.•Expertise in writing Hadoop Jobs for analyzing data using Hive QL (Queries), Pig Latin (Data flow language), and custom MapReduce programs in Java. •Created the Data Marts in dimensional data modeling using star and snowflake schemas. •Used Git for version control with Data Engineer team and Data Scientists colleagues. Involved in creating Created Tableau dashboards using stack bars, bar graphs, scattered plots, geographical maps, Gantt charts etc. using show me functionality. Dashboards and stories as needed using Tableau Desktop and Tableau Server. •Ingested data and performs transformations using ADF and load Cosmos DB using Azure functions (C#).•Worked on converting Hive/SQL queries into Spark Transformations using Spark RRD’s and Python.•Built APIs that will allow customer service representatives to access the data and answer queries. •Extending the functionality of Hive with custom UDF s and UDAF's. •The new Business Data Warehouse (BDW) improved query/report performance, reduced the time needed to develop reports and established self-service reporting model in Cognos for business users. •Implemented Bucketing and Partitioning using hive to assist the users with data analysis. •Developed story telling dashboards in Tableau Desktop and published them on to Tableau Server which allowed end users to understand the data on the fly with the usage of quick filters for on demand needed information.•Created numerous pipelines in Azure using Azure Data Factory v2 to get the data from different source databases Informix, Sybase etc by using different Azure Activities like Move &Transform, Copy, filter, for each, Databricks etc.• Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL.

•Created consumption views on top of metrics to reduce the running time for complex queries. •Compare the data in a leaf level process from various databases when data transformation or data loading takes place. Needed to analyze and performed data quality when these types of loads are done, to look for any data loss, data corruption. •As a part of Data Migration, wrote SQL Scripts for Mismatch of data and worked on loading the history data from Teradata SQL to snowflake. •Created Metric tables, End user views in Snowflake to feed data for Tableau refresh. •Generated Custom SQL to verify the dependency for the daily, Weekly, Monthly jobs. •Created performance dashboards in Tableau/ Excel / Power point for the key stakeholders •Incorporated predictive modeling (rule engine) to evaluate the Customer/Seller health score using python scripts, performed computations, and integrated with the Tableau viz. •Developed SQL scripts to Upload, Retrieve, Manipulate, and handle sensitive data (National Provider Identifier Data i.e Name, Address, SSN, Phone No) in Teradata, SQL Server Management Studio and Snowflake Databases.•Implemented Defect Tracking process using JIRA tool by assigning bugs to Development Team •Involved in Functional Testing, Integration testing, Regression Testing, Smoke testing and performance Testing. Tested Hadoop Map Reduce developed in python, pig, Hive •Developed spark code and spark-SQL/streaming for faster testing and processing of data. •Analyzed marketing campaigns from various perspectives including CTR, seasonal/geographical trends, search queries, landing page, conversion funnel, quality score, competitors, distribution channel, etc. to achieve maximum ROI for clients. •Evaluated the performance of Daily deals PLA ads and compare those items with non-daily deal items to see the possibility of increasing ROI. •Experienced in working with spark ecosystem using Spark SQL and Scala queries on different formats like text file, CSV file.

• Building, and maintenance of ETL processes using Hadoop.• Big Data Development and Analytics using Hadoop stack (HDFS, Hive), Spark, Kafka, PySpark. • Define company data assets and develop spark jobs to populate data models. • Development and maintenance of ETL pipelines, metadata definitions and models, queries and reports, scheduled query jobs. • Data cleansing and loading using Python scripting. • Transfer Data from SQL Server and Oracle, DB2 and Teradata DB to Hadoop using SQOOP. • Analyzed structured, semi-structured and structured data flow. • Stored data in No SQL database such as Hbase.• Transfer Data from SQL Server, Oracle, MySQL to Hive using SQOOP. • Advanced knowledge of SQL and query optimization concepts (HIVE, TSQL and/or PL/SQL) • Unix Shell Scripting, including performance tuning, working with Dataframes, Resilient Distributed Dataset (RDD) and code optimization. • AWS experience working with EC2, EMR, S3, Redshift/RDS. • Working proficiency in SDLC CI/CD Execution (GitHub, Jenkins), Data orchestration using airflow

• Performed various Data mapping activities from Source systems. • Performed Data profiling procedures on new sources prior to developing new subject areas in the warehouse. • Assisting with regular data analysis operations and capturing day-to-day findings. • Assisted in data modeling, data mining, Machine Learning methodologies, advanced data processing, ETL optimization. • Followed Best practices for Data visualization for effective message delivery to Target Business Audience. • Worked on Data Governance, which includes Metadata management, Master data Management, Data Quality, Data Security. • Created the Data Marts in dimensional data modeling using star and snowflake schemas. • Participated in regular QA on developed code, categorizing bugs depending on severity.