Nitish T Email & Phone Number

Moline, Illinois, United States

• Developing various functionalities for the Data Ingestion framework using Python, resulting in a 30% reduction in data processing time.• Creating the JSON configuration files to cleanse the data through the generic Spark framework.• Developed Spark SQL Jobs and processed data in AWS S3 with complex business requirements for Data Marts creation for various markets.• Developed A/B testing framework to evaluate model improvements, boosting user satisfaction scores by 15%.• Pioneered the integration of AWS Glue for workflow automation, leading to a 25% increase in efficiency and timely data delivery.• Conceptualized and executed complex Spark SQL Jobs, achieving a 40% improvement in processing large-scale data sets for Data Marts.• Successfully implemented SCD2 logic using Spark for master reference data, enhancing historical data accuracy by 15%.• Good exposure in writing SQL queries in data analysis and fixing data issues for various facts and • dimensions for data warehouse and data marts for star schema and snowflake schema.• Managed data migration from Teradata to Snowflake, facilitating improved data analytics capabilities.• Utilized AWS services such as EMR to run Hadoop and Spark. Jobs, S3 for data storage, and CloudWatch for monitoring. Experienced in AWS Lambda, Athena, Glue, and RDS.• Spearheaded the integration of Apache Airflow and ML flow into CI/CD pipelines, automating model versioning and updates, boosting operational efficiency by 30%, and reducing manual errors.

India

• Developed SQL scripts, securely accessing database credentials from Azure Key Vault to query, manipulate, and retrieve data, adhering to stringent data security standards on Azure Data Factory.• Designed and implemented a wide range of ETL pipelines using Azure Databricks and PySpark to extract, transform, and load data from various sources, including flat files, databases, and APIs.• Developed Azure data factory pipelines by creating datasets, source, and destination-linked services to move the data from the Oracle database to Azure Data Lake Store Raw Zone and used Azure Databricks for performing data transformations and loading data into Azure Data Lake.• Have good experience working with Azure BLOB and Data Lake storage, loading data into Azure SQL Synapse analytics (DW).• Used JIRA to track project progress, manage tasks, and coordinate with team members.• Used Hive for querying and managing large datasets in Hadoop environments.• Configured and monitored Azure Event Hubs and Stream Analytics for real-time data streaming.• Leveraged Azure DevOps for CI/CD pipelines in data engineering workflows.• Used Jenkins for continuous integration and deployment of data applications.• Containerized applications using Docker to ensure consistent deployment across environments.• Designed SSRS reports with sub-reports, dynamic sorting, and defining data sources and subtotals for the report.• We leaned predictive modelling and machine learning techniques to develop our commendation system, which resulted in a 20% increase in user engagement.

India

• Followed Agile methodologies with JIRA for managing sprints, tracking tasks, and collaborating on project development.• Analyzing the S2TM with table descriptions, schema details, and data types for both source and target while mapping the columns, transformations, and the Teradata target table structure.• Ingested data from various sources, including JSON, CSV, and Avro files, into Hadoop and cloud storage systems.• Implemented deep learning techniques for image and text data, including convolutional neural networks (CNN) and recurrent neural networks (RNN).• Built real-time data pipelines using Kafka and Spark Streaming for processing streaming data.• Migrated data from on-premises RDBMS systems to Azure Data Lake and Snowflake.• Prepared the Source and target DDLs based on S2TM and created tables accordingly in Hive and HBase.• Processedboththehistoryanddailyload/incrementaldatawiththeHadoopjobsandwroteShellscriptswithinthejobs.• ExtracteddatafromdatabasesintoHDFSusingSqoopjobswithincremental/fullload.• Solved performance issues in Spark scripts and Hive Scripts with an understanding of Joins, Group, and aggregation.• Developed Oozie workflows for scheduling and orchestrating ETL processes and documented design and unit test cases for ETL processes.• Developed and deployed applications using significant Hadoop ecosystem components, including HDFS, MapReduce, YARN, and more.• Collaborated with Q/A teams to address defects and monitored production jobs.

Master Of Science - Ms, Biomedical And Health Informatics

Bachelor Of Science - Bs, Biotechnology, 6.5

Bachelor Of Science - Bs, Biotechnology