Pranay Rahul Darsi Email & Phone Number

United States

Highly Involved in Data Architecture and Application Design using Cloud and Big Data solutions on AWS, Microsoft Azure. Leading the effort for migration of Legacy-system to Microsoft Azure cloud-based solution. Re-designing the Legacy Application solutions with minimal changes to run on cloud platform. Extract Transform and Load data from Sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake Analytics . Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Databricks. Experience in GCP Dataproc, GCS, Cloud functions, Big Query. Experience in moving data between GCP and Azure using Azure Data Factory. Built the data pipeline using Azure Service like Data Factory to load the data from Legacy SQL server to Azure Data Base using Data Factories, API Gateway Services, SSIS Packages, Talend Jobs, custom .Net and Python codes. Ingested events data from AWS Kinesis into Databricks Delta Lake which is mounted on AWS S3 buckets. Spearheaded the design and development of ETL pipelines using AWS Glue ETL, extracting data from diverse sources and transforming it into the Glue Data Catalog for efficient data processing and querying. Applied transformations on delta tables and copied the data into Snowflake for further analysis. Developed data warehouse model in snowflake for over 100 datasets using whereScape. Creating Reports in Looker based on Snowflake Connections. Build data pipelines in airflow in GCP for ETL related jobs using different airflow operators. Used cloud shell SDK in GCP to configure the services Data Proc, Storage, Big Query. Built Azure Web Job for Product Management teams to connect to different APIs and sources to extract the data and load into Azure Data Warehouse using Azure Web Job and Functions. Loaded the tables from the azure data lake to azure blob storage for pushing them to snow flake.

United States

Worked on development of data ingestion pipelines using ETL tool, Talend & bash scripting with big data technologies including but not limited to Hive, Impala, Spark, Kafka, and Talend. Experience in developing scalable & secure data pipelines for large datasets. Gathered requirements for ingestion of new data sources including life cycle, data quality check, transformations, and metadata enrichment. Define virtual warehouse sizing for Snowflake for different type of workloads. Supported data quality management by implementing proper data quality checks in data pipelines. Worked on Oracle Databases, RedShift and Snowflakes Define virtual warehouse sizing for Snowflake for different type of workloads. Delivered data engineer services like data exploration, ad-hoc ingestions, subject-matter-expertise to Data scientists in using big data technologies. Build machine learning models to showcase Big data capabilities using Pyspark and MLlib. Enhancing Data Ingestion Framework by creating more robust and secure data pipelines. Implemented data streaming capability using Kafka and Talend for multiple data sources. Worked with multiple storage formats (Avro, Parquet) and databases (Hive, Impala, Kudu). S3 - Data Lake Management. Responsible for maintaining and handling data inbound and outbound requests through big data platform. Working knowledge of cluster security components like Kerberos, Sentry, SSL/TLS etc. Involved in the development of agile, iterative, and proven data modeling patterns that provide flexibility. Knowledge on implementing the JILs to automate the jobs in production cluster. Troubleshooted user's analyses bugs ( JIRA and IRIS Ticket). Worked with SCRUM team in delivering agreed user stories on time for every Sprint. Worked on analyzing and resolving the production job failures in several scenarios.

Texas, United States

Designed, deployed, maintained, and lead the implementation of Cloud solutions using Microsoft Azure and underlying technologies. Migrating Services from On-premises to Azure Cloud Environments. Collaborate with development and QA teams to maintain high-quality deployment. Deployed the initial Azure components like Azure Virtual Networks, Azure Application Gateway, Azure Storage and Affinity groups. Conducted capacity planning and architecture of Storage Accounts. Configured Azure Traffic Manager to build routing for user traffic Infrastructure Migrations: Drive Operational efforts to migrate all legacy services to a fully Virtualized Infrastructure. Prepared capacity and architecture plan to create the Azure Cloud environment to host migrated IaaS VMs and PaaS role instances for refactored applications and databases. Installing the Applications Insights tool on the web services and configuring an Application Insight workspace in Azure. configured Application Insights to perform web tests and alerts. Handle deployment workflow including provisioning, load balancing, and health monitoring. Implemented HA deployment models with Azure Classic and Azure Resource Manager. Configured Azure Active Directory and managed users and groups. Configure continuous integration from source control, setting up build definition within Visual Studio Team Services (VSTS) and configure continuous delivery to automate the deployment of ASP.NET MVC applications to Azure web apps. Deployed DSC from Azure Automation to On-premises or cloud environments. Installing IIS and ASP.NET modules, configuring server and adding ASP.NET Applications via command line. Configured Azure Automation DSC configuration management to assign permissions through RBAC, assign nodes to proper automation accounts and DSC configurations, to get alerted on any changes made to nodes and their configuration.

India

Used custom developed PySpark scripts to pre-process, transform data and map to tables inside the CIF (Inmon Corporate Information Factory) data warehouse. Developed shell scripts of Sqoop jobs for loading periodic incremental imports of structured data from various RDMS to S3 and used Kafka to ingest real-time website traffic data to HDFS. As part of reverse engineering discussed issues/complex code to be resolved and translated them into Informatica logic and prepared ETL design documents. Experienced working with team, lead developers, interfaced with business analysts, coordinated with management, and understand the end user experience. Used Informatica Designer to create complex mappings using different transformations to move data to a Data Warehouse. Developed mappings in Informatica to load the data from various sources into the Data Warehouse using different transformations like Source Qualifier, Expression, Lookup, aggregate, Update Strategy and Joiner. Optimized the performance of the mappings by various tests on sources, targets and transformations. Scheduling the sessions to extract, transform and load data into warehouse database on Business requirements using scheduling tool. Extracted (Flat files, mainframe files), Transformed and Loaded data into the landing area and then into staging area followed by integration and sematic layer of Data Warehouse (Teradata) using Informatica mappings and complex transformations (Aggregator, Joiner, Lookup, Update Strategy, Source Qualifier, Filter, Router and Expression Optimized the existing ETL pipelines by tuning SQL queries and data partition techniques. Created independent data marks from existing data warehouse as per the application requirement and updated them on bi-weekly basis. Decreased the Azure billing by pivoting from using Redshift storage to Hive tables for unpaid services and implemented various techniques like Partitioning and Bucketing over hive tables to improve the query performance.

Master Of Science - Ms, Computer Science

Bachelor Of Technology - Btech, Computer Science