Ugandar Hari Email & Phone Number

Melbourne, Vic, Au

Melbourne, Victoria, Australia

• Enhancing ETL processes by ingesting the data related to Novated Leasing, Salary Packaging and Fleet Management into Azure Data Lake from multiple source applications such as SQL Server, Azure SQL Database and JSON files, transforming the data using DBT Cloud and loading them into Snowflake DB.• Incorporated new source from Azure SQL Database by creating new linked service, creating new pipelines and loaded the raw data into Snowflake Database.• Incorporated new columns from SQL Server to the existing table in Snowflake DB – end to end by executing the recovery process to update the ConfigDB, pointing the database to the recovery tables and triggered the ADF pipeline to perform full load and CDC data load.• Transformed the data using DBT Cloud, by creating DBT models, tests and snapshots and loaded the transformed data into Snowflake Database.• Created DBT macros to perform generic and custom tests to test the data issues, unique, not-null, duplicate issues, to generate merge query for incremental load and masking policies for PII data (Personally identifiable information).• Deployed the Azure Data Factory pipelines (ARM templates) and DBT models using Azure DevOps by creating feature branches and pull requests.• Upgraded deprecated legacy Snowflake linked services, which is connected via SelfHostedIR and created global parameters to pass it to the linked service through datasets from the pipelines.• Created Key Vault Secrets for Snowflake DB passwords and DBT Secret tokens.• Created requests to add my account into Managed Identity for Azure resource manager and Azure SQL Database.• Cleaned up the existing ADF pipelines to remove redundant pipelines and its datasets, continuously monitoring the pipelines and jobs, improving the job performance and deploying the hot fixes in production environment.

Australia

• Integrated Databricks with DBT Core, transformed the data using Models and loaded the transformed datainto Databricks SQL Warehouse.• Worked in Singular and Generic Tests to perform the query testing before loading the data into DWH.• Invoked Macros from the DBT Models to convert the date columns and to find the difference between twodates.• Created Jobs and Delta Live Tables to build Lakehouse architecture in Databricks and managed taskorchestration.• Migrated the code changes using Databricks repos to higher regions.• Coordinated with cross applications platforms such as FFL, SDWX and SDS2 to understand the regressiontesting efforts.• Coordinated with the testing team and solutions architect to come up with the list of tables impacted andreviewed the test case documents.• Booked and secured the environments for Development and Test regions required for delivery scheduleddates.• Involved in data preparation for data flow from multiple applications to SDSX.• Supported the testing team for baseline setup and helped in accomplishig regression testing.

Melbourne, Victoria, Australia

• Interacted with stakeholders to define project goals, scope, performance requirements, timelines and deliverables.• As a Technical Lead, I have been involved in design discussions, allocated tasks to Data Engineers, performed code reviews and made sure my team members are on the same page.• Mentored and trained the Data Engineers, supported them to grow their technical skills and understanding of best practices.• Created an End to End pipeline in Azure Synapse Analytics workspace to extract, transform and load.• Extracted the data from source system applications (On-premises SQL Server) and ingested into Azure Data Lake Gen2 using Data Factory pipeline.• Transformed the data using Databricks Notebooks and converted the data into delta file format in the Spark pool.• Stored secret keys in Key Vault secrets and used it in Data Factory to access SQL Server and Databricks .• Loaded the data into dedicated SQL pool database for data analysis and reporting purpose based on business users' need.• Performed validation in ADF pipelines and created alerts for log analytics query output for validation status.• Scheduled and executed the pipeline in Azure Synapse Analytics using triggers in daily and weekly frequencies.• Monitored SQL pools and Spark pools to see the query performance, memory allocations etc.• Built CI/CD pipeline in Azure Devops repositories to integrate Synapse Analytics Dev workspace, created feature branches based on the master branch and deployed the code from DEV to UAT to PROD.• Involved in unit testing, functional testing, system integration testing and implementation activities.

Adelaide, South Australia, Australia

• Worked in Azure resources including Azure Data Lake, Data Factory, Databricks, Synapse Analytics andPower BI.• Created Azure Data Factory pipeline and ingested the data from SQL Server On-premises database andloaded the data into Data Lake Storage Gen2.• Worked on reading multiple source data formats such as SQL database, csv and text files and convertedinto delta file format.• Integrated Databricks Notebooks into Data Factory pipeline, transformed the data using PySpark scriptsto create type2 dimensions and transaction data and loaded them into Data Lake Storage.• Created tables in Snowflake, loaded the data using Azure Data Factory into Snowflake database, analysedthe data by creating storage integration, stage and executed copy command to insert the records.• Scheduled the Data Factory pipeline using triggers to automate the ingestion, transformation and loadingof delta data into Data Lake Storage and Snowflake database.• Created serverless SQL database views using stored procedure in Synapse Analytics to analyse thetransformed data.• Configured Power BI to connect to serverless SQL database of Synapse Analytics and analysed the datawith the help of Data Analyst to generate graphical representation of the target data to the end users.• Built CI/CD pipeline in Azure Devops repositories to integrate Databricks Dev workspace and created featurebranches based on the main branch and deployed the notebooks from DEV to UAT to PROD.

Chennai, Tamil Nadu, India

• Interacting with clients, requirements gathering and preparing requirement specification and design documents for project deliverables.• Migrated existing on-premises application to Microsoft Azure.• Created Data Factory pipelines to ingest the data from the source, executed Databricks instances to transform the data using PySpark scripts.• Loaded the transformed data into SQL Database for reporting purpose based on business users' need.• Analysed the data by connecting the SQL Database in MS SQL Server Management Studio and updated the PySpark code based on the query output.• Managed Azure Active Directory to create security groups and added users in the groups to give resource groups access to the security groups using IAM roles assignment.• Involved in ETL (Extract-Transform-Load) migration projects from Datastage v9.1 to v11.5 and v11.5 to v11.7.• Extracted data from source system applications, transformed the data using sequencers and jobs in Datastage Designer and monitored the jobs in Director according to the requirement.• Developed Unix Shell scripting for file validation, file comparison and generic script for reading different parameter files.• Involved in continuous build and integration, unit testing, system testing, implementation activities and performance tuning of ETL processes.

Chennai, Tamil Nadu, India

• Analysed the specification document and Mainframe jobs.• Involved in initial study of system, coordinated requirements gathering sessions with business analysts, end users and people from other applications.• Involved in impact analysis, preparation of requirement specifications and design documents (High Level and Low Level Design) for project deliverables.• Involved in source to target mapping document and migration of Mainframe batch components to the ETL target system.• Developed sequencers and jobs in Datastage Designer and a process to check if source and target are in sync.• Scheduled the jobs and executed them using Autosys.• Involved in unit testing, system testing and implementation activities and performance tuning of ETL processes.

India, Usa