Diego Lopes work email
- Valid
Diego Lopes personal email
I have been a data engineer for five years, with solid experience designing and implementing scalable data solutions for batch processing when in real-time. I am currently studying for a master's degree in Computer Science at USP, my research is focused on Generative AI and Natural Language Processing (NLP) to improve Text-to-SQL techniques for geospatial data queries.I have a solid foundation in relational and non-relational databases. Which included PostgreSQL, MySQL, Cassandra, and MongoDB. I have experience in data modeling, and pipeline development using tools such as Apache Spark and Airflow, as well as architecting data lakes for efficient data storage and retrieval and data warehouses for data analysis. Additionally, I have hands-on experience in data governance and security, ensuring data integrity, confidentiality, and availability in compliance with relevant regulations and industry best practices.
-
Senior Data EngineerPulsusSão Paulo, Sp, Br -
Deep Learning Engineer & Researcher - Master’S DegreeUsp - Universidade De São Paulo Jul 2023 - PresentSão Paulo, BrazilThis project aims to improve the CulturaEduca platform through the implementation of a naturallanguage interface that allows non-expert users to access geospatial data using Text-to-SQLtechniques. Our focus is to improve translation results in geospatial queries, which is currentlychallenging due to the limited representation of complex queries on the training datasets ofcurrent Text-to-SQL methods. We will conduct this project as a case study, using Generative AIand… Show more This project aims to improve the CulturaEduca platform through the implementation of a naturallanguage interface that allows non-expert users to access geospatial data using Text-to-SQLtechniques. Our focus is to improve translation results in geospatial queries, which is currentlychallenging due to the limited representation of complex queries on the training datasets ofcurrent Text-to-SQL methods. We will conduct this project as a case study, using Generative AIand Processing Natural Language (NLP) to achieve our goals. Show less -
Senior Data EngineerPulsus Aug 2022 - PresentPorto Alegre, Rio Grande Do Sul, BrazilIn this role, I am responsible for designing the data architecture to process data in real-time andbatch with an average volume of 10 million 60kb documents daily. My responsibilities alsoinclude understanding the needs and challenges of the business to implement an architecturethat can handle large volumes of data per second, ensuring data quality, security, and availabilitywhile ensuring compliance with data protection regulations.Technologies: Apache Airflow, Apache… Show more In this role, I am responsible for designing the data architecture to process data in real-time andbatch with an average volume of 10 million 60kb documents daily. My responsibilities alsoinclude understanding the needs and challenges of the business to implement an architecturethat can handle large volumes of data per second, ensuring data quality, security, and availabilitywhile ensuring compliance with data protection regulations.Technologies: Apache Airflow, Apache Cassandra, Grafana, MongoDB, Python, SQL, AWS (S3,RDS, EC2, Athena, Glue) e PostgreSQL. Show less -
Senior Data EngineerAct Digital May 2022 - Aug 2022BrazilIn this position, I was responsible for data management and was responsible for creating andoptimizing the data flow responsible for consolidating the user journey between platforms fromGrupo Globo. For this, I was responsible for maintaining the architecture of the databases andpipelines that processed data coming from a graph database and other databases for ananalytical basis.Technologies: AWS (Neptune), GCP (BigQuery e Dataform), JavaScript, Python -
Data EngineerCi&T Mar 2021 - Apr 2022Campinas, São Paulo, BrazilIn this project, I was responsible for making the connection between the business teams thatdedicated themselves to implementing the Salesforce interface for Account Managers and thesales team data at Bradesco Bank. My responsibilities were focused on analyzing the prototypesdeveloped and evaluating the feasibility and availability of the data necessary to implement thenew features. Performed data mapping, identification of databases available such as Teradata,SQL Server, or… Show more In this project, I was responsible for making the connection between the business teams thatdedicated themselves to implementing the Salesforce interface for Account Managers and thesales team data at Bradesco Bank. My responsibilities were focused on analyzing the prototypesdeveloped and evaluating the feasibility and availability of the data necessary to implement thenew features. Performed data mapping, identification of databases available such as Teradata,SQL Server, or Hive, as well as column and rule mapping to perform the metrics andnormalizations of the raw layered data lake refined, in addition to data normalization.Technologies: Apache Hive, Erwin, Python, Teradata, SQL Server, SQL Server Integration Service. Show less -
Data EngineerEy Sep 2020 - Mar 2021Sao Paulo, Sao Paulo, BrazilIn this position, I was responsible for implementing new financial reports for products of creditfor Bradesco Bank, and I was responsible for performance optimization and refactoring of existingHiveQL scripts. My improvements resulted in a 20-minute reduction in the 1-hour run time ofreports. I was responsible for monitoring ELT processes to ensure the execution and delivery ofdata.Technologies: Apache Hive e HiveQL, Shell Scripting, Linux. -
Data EngineerMovile Aug 2018 - May 2020Campinas Area, BrazilIn this project, I worked with two distinct segments: applications and messaging, including SMSand WhatsApp Bots. My responsibilities included maintaining data pipelines and dashboardscompany metrics and identifying the needs of the marketing, product, and finance teams. In theapplications area, I consolidated metrics such as DAU, WAU, and MAU to create the user journeyinside the product. Created data flows using Pentaho Data Integration, which ran scripts SQL thatperformed… Show more In this project, I worked with two distinct segments: applications and messaging, including SMSand WhatsApp Bots. My responsibilities included maintaining data pipelines and dashboardscompany metrics and identifying the needs of the marketing, product, and finance teams. In theapplications area, I consolidated metrics such as DAU, WAU, and MAU to create the user journeyinside the product. Created data flows using Pentaho Data Integration, which ran scripts SQL thatperformed all data normalization and metrics calculation steps. In the area of messaging, Iworked on consolidating the data warehouse in Google Big Query and Amazon Redshift inaddition to developing new data flows using Apache Airflow and Python to process orchestration.I was responsible for carrying out data enrichment through Credit information APIs, such asSerasa Score. Additionally, I maintained and created new dashboards in Grafana and Power BIthat consumed data from Elasticsearch and Google BigQuery respectively, to monitor the trafficof messages sent and delivered to users.Technologies: Apache Airflow, AWS (Redshift), GCP (BigQuery), Python, SQL Server, Postgres,Power BI e Pentaho Data Integration. Show less
Diego Lopes Education Details
-
Computer Science -
Digital Security, Governance And Data Management -
Analysis And Systems Development
Frequently Asked Questions about Diego Lopes
What company does Diego Lopes work for?
Diego Lopes works for Pulsus
What is Diego Lopes's role at the current company?
Diego Lopes's current role is Senior Data Engineer.
What is Diego Lopes's email address?
Diego Lopes's email address is di****@****us.mobi
What schools did Diego Lopes attend?
Diego Lopes attended Usp - Universidade De São Paulo, Pontifícia Universidade Católica Do Rio Grande Do Sul, Instituto Federal De Educação, Ciência E Tecnologia De São Paulo - Ifsp.
Who are Diego Lopes's colleagues?
Diego Lopes's colleagues are Letícia Oswald, Rosa Alice Parente, Willian Comar, Emanuel Rodrigues, Felippe Menezes, Automobile Engineering, Guilherme Meloni.
Not the Diego Lopes you were looking for?
-
-
2fesa.com.br, fesagroup.com
-
-
Diego Lopes
Sales Management | Precision Ag | Product Marketing | Tactical Marketing | AgribusinessMogi Das Cruzes, Sp
Free Chrome Extension
Find emails, phones & company data instantly
Aero Online
Your AI prospecting assistant
Select data to include:
0 records × $0.02 per record
Download 750 million emails and 100 million phone numbers
Access emails and phone numbers of over 750 million business users. Instantly download verified profiles using 20+ filters, including location, job title, company, function, and industry.
Start your free trial