Ashwin D Email & Phone Number

United States

• Transformed business problems into Big Data solutions and define Big Data strategy and Roadmap. • Installed, configured, and maintained Data Pipelines• Designed the business requirement collection approach based on the project scope and SDLC methodology.• Created Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform, and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.• Worked with Data governance and Data quality to design various models and processes.• Involved in all the steps and scope of the project reference data approach to MDM, have created a Data Dictionary and Mapping from Sources to the Target in MDM Data Model.• Automated the data processing with Oozie to automate data loading into the Hadoop Distributed File System.• Designed and Developed Oracle PL/SQL and Shell Scripts, Data Import/Export, Data Conversions and Data Cleansing. • Performed data analysis, statistical analysis, generated reports, listings, and graphs using SAS tools, SAS/Graph, SAS/SQL, SAS/Connect and SAS/Access.• Developed Spark applications using Scala and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats.• Used kafka and integrating with the Spark Streaming• Authored Python (PySpark) Scripts for custom UDF’s for Row/ Column manipulations, merges, aggregations, stacking, data labeling and for all Cleaning and conforming tasks.• Used ORC, Parquet file formats on HDInsight, Azure Blobs and Azure tables to store for raw data.• Involved in writing T-SQL working on SSIS, SSAS, Data Cleansing, Data Scrubbing and Data Migration.• Worked on Dimensional and Relational Data Modeling using Star and Snowflake Schemas, OLTP/OLAP system, Conceptual, Logical and Physical data modeling using Erwin. • Performed PoC for Big data solution using Hadoop for data loading and data querying

• Expertise in designing and deployment of Hadoop cluster and different Big Data analytic tools including Pig, Hive, HBase, Oozie, Sqoop, Flume, Spark, Impala. • Ingested the data from Relational Databases to HDFS using SQOOP • Implemented advanced procedures like text analytics and processing using the in-memory computing capabilities like Apache Spark written in python • Implemented Spark using python and Spark SQL for faster testing and processing of data. • Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, Scala. • Worked with Spark to create structured data from the pool of unstructured data received. • Implemented intermediate functionalities like events or records count from the flume sinks or Kafka topics by writing Spark programs in java and python. • Chosen and produced information into csv records and put away them into AWS S3 by utilizing AWS EC2 and afterward organized and put away in AWS Redshift. • Developed Glue Jobs to read data in csv format in raw layer and write data to parquet format in publish layer.• Developed a Glue job which does delete, update and incremental loads from source to target.• Extract Real time feed using Kafka and Spark Streaming and convert it to RDD and process data in the form of Data Frame and save the data as Parquet format in HDFS. • Experienced in transferring Streaming data, data from different data sources into HDFS, No SQL databases • Created ETL Mapping with Talend Integration Suite to pull data from Source, apply transformations, and load data into target database.• Creating Databricks notebooks using SQL, Python and automated notebooks using jobs.• Creating Spark clusters and configuring high concurrency clusters using Azure Databricks to speed up the preparation of high-quality data.• Used PySpark and Pandas to calculate the moving average and RSI score of the stocks and generated them into data warehouse.

• Worked on code migration of quality monitoring tool from Amazon EC2 to AWS Lambda and built logical datasets to administer quality monitoring on snowflake warehouses.• Implemented and setup AWS shield, AWS, config, Amazon Macie, and Amazon inspector for security and protection of sensitive data.• Automation of cloud infrastructure using Terraform, and application configuration and deployment.• Creating and managing access to AWS services for IAM user accounts and for role-based users.• Using Tableau, designing dashboard to show operational metrics.• Hands-on experience integrating AWS services: EC2, S3, Network Protocol, Transit VPC, VPC Peering, VPC Endpoints, VPC Private Link.• Developed multiple POC’s using Spark, Scala and deployed on the Yarn Cluster, compared the performance of Spark, with Hive and SQL.• Used Amazon Elastic Cloud Compute (EC2) infrastructure for computational tasks and Simple Storage Service (S3) as storage mechanism.• Evaluated Snowflake design considerations for any change in the application.• Implemented Installation and configuration of the multi-node cluster on Cloud using AWS.• Experienced with Spark Streaming and AWS Kinesis for real-time data processing.• Configured the services S3, AWS Glue, EC2 using python Boto 3. • Implemented ETL pipelines in AWS Glue for various spark transformations.• Involved in PL/SQL query optimization to reduce the overall run time of stored procedures. • Worked on Kibana dashboards based on log stash data and integrated several source and target systems into Elastic search for near real-time log analysis of end-to-end transaction monitoring.• Integrated Apache Airflow with AWS to monitor multi-stage machine learning processes with Amazon Sage Maker jobs.

• Utilized Spark SQL to perform advanced-level data extraction, data transformation, data management tasks providing on the go responses to some management questions by performing complex joins, queries. • Responsible for fully documenting, managing library of source code, algorithms for future use.• Developed, tested hypotheses (t-test, F-test) using R to support research, product offerings and communicate findings to data reports/ visualization in a clear, precise, actionable manner using tools such as Looker, Power BI and Tableau.• Extensive practical knowledge in importing data for use in reporting software (Looker/Tableau) graphs• Pulling ad-hoc reporting SSRS, sales force customize to client’s need by using Alteryx , advance Excel. • Creating and maintaining various reports, tracking spreadsheets, and on-demand ad-hoc report requests, as needed, and within specific timeframes.Experience in developing Tableau and Looker dashboards.• created new OLAP data mart for more concise data and more efficient report performance. • Created KPI, Partitions, Perspectives, and Calculated Measures, used DAX queries in Power BI.

Master Of Science - Ms, Business Analytics

Bachelor Of Technology - Btech, Electrical, Electronics And Communications Engineering