Data Engineer
CurrentAllocated to client AB-Inbev.Project - BrewGen:Operating on a new global system for managing world-class agricultural research programs, which includes fully integrated data storage, to facilitate and optimize variety development for malting and brewing.Applied technologies:- Azure SQL- Azure Data Lake Storage- Azure Databricks- Azure Data Factory- Azure Functions Serverless- Python with Pandas and PySpark- PowerBIMain activities:Organization of the lakehouse, preparation of bronze, silver and gold layers, cleaning and optimization of processing, automation of synchronizations with data factory and serverless functions, 90% reduction in time and costs in the cloud with intelligent processing that identifies changed data by processing only what you need, delivering reliable data to generate insights in PowerBI for decision making by management and directories.Modeling a new database for managing global seed stocks, being able to edit quality, transfer seeds between programs, reset stock positions, merge and split seed bags.