Allan Farias Email & Phone Number

João Pessoa, Pb, Br

As a Data Engineer, I play a fundamental role in building and optimizing data solutions using the Azure ecosystem. My responsibilities include:– Development of Data Pipelines: Designing and implementing ETL (Extract, Transform, Load) pipelines using Azure Data Factory, orchestrating data flows between various sources and destinations.– Data Processing and Transformation: Using Azure Databricks to process large volumes of data, applying pre-processing and data cleaning techniques with PySpark and Python. Transforming data into optimized formats such as Parquet and Delta Tables.– Process Automation: Contributing to the automation of data processes, including file ingestion and report generation. Using Azure Logic Apps and Python scripts to automate the sending of notifications and email reports.– Data Validation and Quality Assurance: Developing and implementing data validation and quality rules to ensure the accuracy of processed information. Executing quality tests to ensure compliance with business rules.– Systems Integration: Integrating data from various systems, using cutting-edge technologies to consolidate information and facilitate advanced analyses.– Data Documentation and Modeling: Preparing technical documentation for ETL processes and data validation rules. Contributing to data table modeling, creating optimized structures for analysis in data warehouses.– Collaboration in Multidisciplinary Teams: Working with development teams, business analysts, and other stakeholders to ensure the delivery of high-quality data solutions. Participating in planning meetings and sprint reviews in agile environments.

Postgraduate Degree, Data Engineer

Sistemas Para Internet

Graduando, Human Nutrition

Administração