Focusing on data processing and consumption, I have been exploring solutions to enhance the performance of our data platform. By integrating AWS with lakehouse services such as Databricks and Snowflake, we aim to optimize our data management and analytics capabilities.

I am a part of a team that works to develop data products, working on receiving, process and making available the most diverse data sources (internal and external APIs, SQL and NoSQL Databases, FTP). I was the main engineer on the data lake Migrate project, leaving a flow with Pentaho Kettle and Redshift and going to a Data Lake structure inside the AWS Cloud. Our main data source is an Oracle with more than a thousand tables and almost 2 terabytes, and by using AWS DMS solution, we brought all this data to AWS S3 creating our Raw Zone. All processing between the Raw and Processed layers was performed by EMR (PySpark) and orchestrated by Airflow. The Processed Zone is the first accessible layer on Data Lake and we use AWS Athena as the SQL engine. In the Curated and Analytic layers, we implemented the Data-Mesh concept, where the business areas are responsible for the rules of each entity within this micro-universe of data. Aiming to obtain more autonomy for the business areas, we use DBT (Data Build tools) as a framework to manage, orchestrate and implement all tables up to the Analityc Zone.

I worked in large and medium-sized companies, I was responsible for the architecture and implementation of Data Lake projects. Using an AWS cloud, I structured ingest services for APIs, relational and non-relational databases (DMS, Lambdas, Kinesis), Data Lake layer storage (S3) and making the processed and enriched data available to the area (EMR, Redshift, Snowflake, Athena, Presto). All services used in the projects were coded using Terraform and I used jobs Sparks (PySpark) for processing. In other projects I acted as Tech Lead Data Engineer, being responsible for the deliveries of the technical team. Working closer to the customer, identifying as a need, managing the backlog and deadlines, thinking about the solution to the most diverse problems.

I acted as the data engineer responsible for the architecture and development of the data platform. Assisting in data engineering team building. Our data architecture was developed using AWS services such as DMS, ECS, S3, KMS, EMR (PySpark), Glue, Lambda and Redshift. All processes were orchestrated by Airflow and we maintained a data warehouse in Redshift, where we generated insights for the business areas.

Support the development team, application deployments, query performance analysis, and reporting for business areas. Data lake - Development of data pipeline for batch processing, using DMS to collect data from a Postgres database and stored in an S3 bucket with Parquet files. And using AWS Glue Job to transform this data and generate insights for the product team. Development of a python application to perform client migration between the legacy system and the new one.

DBA - Infra e Desenvolvimento.Plataformas:- Oracle- SqlServerAdministração de servidores SQL Server (versões: 2000, 2005, 2008 e 2012). Automatização de processos de checklist, monitoração e manutenção do ambiente de produção, análises de traces, tunning de queries, particionamento de tabelas e execução de restores e backups.. Instalação e configuração do SQL Server Stand-Alone, Clusters, Analysis Services e Reporting Services.Administração de servidores de banco de dados Oracle. Gerencia de usuários, schemas, roles e bloqueios. Desenvolvimento de procedures em Pl/SQL e TSQL para otimizar o trabalho.

Projeto BrasilCAP: Participei da equipe de Sadmin do projeto. Administração do aplicativo Oracle Siebel CRM, Siebel Tools, compilação de SRF, repository merge, deploy de ambiente e aplicação de patches para upgrade de versão. Oracle BI Publisher e integração da barra de CTI com o Siebel.Instalação e configuração do Oracle Business Intelligence Enterprise Edition 11g. (Informatica, Data Warehouse Administration Console e BIApps).

Desenvolvimento de queries e procedures no Oracle. Dominio das ferramentas PL/SQL e Oracle BI Discoverer. Assistencia na análise dos requisitos e documentação dos projetos da area juridica, acompanhamento da execução dos projetos junto as fabricas de softwares. Suporte aos processos do Juridico. Utilização do MS Project.