Nishikanth R. Email & Phone Number

Tampa, Florida, United States

• Designed and implemented scalable data ingestion pipelines using Azure Data Factory, ensuring extraction from diverse sources.• Created Pipelines in Azure ML Workspace and ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards. • Developed end-to-end data integration solutions with Azure Data Factory, orchestrating workflows and loading data into target systems.• Leveraged Azure Databricks for optimizing Spark jobs and integrating diverse data sources for efficient ingestion and transformation.• Engineered ETL pipelines for historical and incremental data transfer to Snowflake database and utilized Azure Blob Storage and Azure Data Lake Storage Gen 2.• Proficient in Azure DevOps for CI/CD workflows, automated pipeline deployments, and version control practices.• Managed Spark Databricks clusters, optimized performance, tuned batch intervals, parallelism, and memory, developed Azure ML pipelines and JSON scripts for Azure Data Factory to process data via SQL activities.• Designed and developed Spark applications using Pyspark and Spark-SQL for data extraction, transformation, and aggregation.• Implemented different pipeline types, including incremental, full load, and historical data loads.• Collaborated effectively with cross-functional teams to ensure the implementation and maintenance of best practices in Azure data engineering solutions

Dallas, Texas, United States

• Demonstrated expertise in Azure Cloud Services, Azure Synapse Analytics, SQL Azure, Data Factory, Azure Analysis Services, Application Insights, Azure Monitoring, Key Vault, and Azure Data Lake for seamless integration and data processing.• Created pipelines, data flows, and complex data transformations and manipulations using ADF and PySpark with Databricks.• Data Ingestion to Azure Data Lake, to Azure DW for data migration processing the data in Azure Databricks.• Used Data Flow debugging for effectively building ADF data flow pipelines. • Improved performance by using optimization options by effectively using partitions during various transformations.• Implemented complex ETL Azure Data Factory pipelines using mapping data flows with multiple Input/output transformations• Developed Spark applications using Spark-SQL in Databricks for data extraction, transformation, and aggregation from diverse file formats, providing insights into customer usage patterns• Used Azure Key vault as a central repository for maintaining secrets and referenced the secrets in Azure Data Factory and also in Databricks notebooks.• Built a common SFTP download or upload framework using Azure Data Factory and Databricks• Hands-on experience with Snowflake utilities, Snow SQL, Snow Pipe, and Big Data model techniques using Python/ Scala.• Responsible for Building and scaling ETL / Event processing systems that organize data and manage complex rule sets in batch and real-time.• Designed ETL pipelines in and out of the data warehouse using a combination of Python and Snowflakes Snow SQL Writing SQL queries against Snowflake

Charlotte, North Carolina, United States

• Designed and implemented end-to-end data pipelines on AWS and Databricks platforms to ingest, process, and analyze large volumes of data for various business use cases.• Utilized AWS Glue for data cataloging, ETL processing, and schema evolution, ensuring data consistency and availability across the organization.• Implemented scalable data lake architectures on AWS S3 and Delta Lake, optimizing storage and query performance for efficient data access and analysis.• Integrated Glue with other AWS services such as S3, RDS, Redshift, and Athena for seamless data processing.• Developed and maintained PySpark applications for data processing, transformation, and machine learning tasks, ensuring high-performance and fault-tolerant data processing.• Migrated on-premises databases to Amazon RDS with minimal downtime using AWS Database Migration Service.• Enforced data security and compliance using AWS IAM roles, encryption, and parameter groups.• Performed bulk loading from the external stage (AWS S3), internal stage to snowflake cloud using the COPY command.• Designed and implemented complex workflows and state machines using AWS Step Functions.• Conducted regular audits and reviews of CloudFormation templates for optimization and best practices.• Implemented event-driven architectures by integrating Lambda with AWS services such as S3, SNS, SQS, and DynamoDB.• Loaded data into snowflake tables from the internal stage using Snow SQL.• Written complex Snow SQL scripts in snowflake cloud data warehouse to business analysis and reporting.• Implemented data governance and security best practices on AWS and Databricks platforms, ensuring compliance with regulatory standards and data privacy policies

Rochester, Michigan, United States

• Created and maintained reporting infrastructure to facilitate visual representation of manufacturing data for purposes of operations planning and • Created continuous integration and continuous delivery (CI/CD) pipeline on AWS that helps to automate steps in software delivery process. • Experience in processing of load and transform the large data sets of structured, unstructured and semi structured data in Hortonworks.• Worked on AWS Data Pipeline to configure data loads from S3 to into Redshift and used Glue to define table and column mapping from S3 data to Redshift. • Worked on Distributed/Cloud Computing (Map Reduce/Hadoop, Pig, HBase, AVRO, Zookeeper), Amazon Web Services (S3, EC2, EMR etc.) • Worked on importing some of the data from NoSQL databases including HBase. • Worked on AWS Data Pipeline to configure data loads from S3 to into Redshift and used Glue to define table and column mapping from S3 data to Redshift. • Worked on Distributed/Cloud Computing (Map Reduce/Hadoop, Pig, HBase, AVRO, Zookeeper), Amazon Web Services (S3, EC2, EMR etc.) • Worked on importing some of the data from NoSQL databases including HBase.• Experience in using Spark framework with Scala and Python. Good exposure to performance tuning hive queries and MapReduce jobs in spark (Spark-SQL) framework on Hortonworks.• Wrote scripts and indexing strategy for a migration to Confidential Redshift from SQL Server and MySQL databases .• Performed transformations, cleaning and filtering on imported data using Hive, MapReduce, and loaded final data into HDFS.

Topeka, Ks

• Experience in developing complex store procedures, efficient triggers, required functions, and creating indexes and indexed views for performance.• Excellent Experience in monitoring SQL Server Performance tuning in SQL Server• Expert in designing ETL data flows using SSIS, Created mappings/workflows to extract data from SQL Server and the Migration and Transformation from Access/Excel Sheets using SQL Server SSIS.• Experience in Error and Event Handling: Precedence Constraints, Break Points, Check Points, Logging.• Thorough knowledge of Features, Structure, Attributes, Hierarchies, and Star and Snow Flake Schemas of Data Marts.• Created paginated reports using Power BI Report Builder for on-premises and cloud data sources.• Developed SSAS Cubes, Aggregation, KPIs, Measures, Partitioning Cubes, Data Mining Models, and Deploying and Processing SSAS objects.• Developed Star and Snow Flake Schemas of Data Marts.• Created Ad hoc reports and reports with complex formulas and querying the database for Business Intelligence.• Expertise in developing Parameterized, Chart, Graphs, Linked, Dashboard, Scorecards, and Report on SSAS Cube using Drill-down, Drill-through, and Cascading reports using SSRS.