Experienced Data Engineer, I am passionate about leveraging data to drive meaningful insights and optimize business processes.I specialize in designing and implementing robust data pipelines on AWS infrastructure, ensuring efficient ETL operations and the availability of high-quality data for analysis. My toolkit includes Snowflake, AWS Glue, and both OLAP and OLTP relational databases, complemented by highly optimized SQL queries. Additionally, I am proficient with Databricks for developing data processing pipelines using both Scala and PySpark. My experience also extends to working with Agile methodologies and version control systems like Git.Recent highlights of my career include developing data warehouses on Redshift that support near-real-time incremental loading, and executing SQL queries in-memory, controlled by Python applications.I am currently pursuing an MSc in Data Management to deepen my knowledge and ensure I remain at the forefront of the data engineering field.
-
Data EngineerOptum Oct 2023 - Aug 2024Los Angeles Metropolitan Area• Modeled the data transformation process in order to standardize different EMR and EHR systems to the Optum platform, allowing the integration of several more services such as Epic Clarity and NAMM EHR.• Implemented databricks pipelines using Scala to execute batch transformations leveraging new business options and accelerating ETL processing on 10 times fold• Modeled queries on Snowflake for Data Analyst and Data Quality teams deliver high-quality data quickly -
Data EngineerLobby Cre Dec 2021 - Oct 2023Charlotte, North Carolina, United States• I designed and implemented a state-of-the-art SQL data lakehouse on Redshift from scratch, unlocking a host of advanced analytics solutions for the real estate industry.• I successfully developed innovative in-memory data processing solutions using DuckDB on an AWS ECS application, reducing the ingestion latency time 10 times fold.• I supported the CS team by providing comprehensive tech support, conducting in-depth data analysis, and closely monitoring AWS cloud infrastructure for prompt issue resolution. -
Data EngineerBluemetrics Jul 2021 - Oct 2023Porto Alegre, Rio Grande Do Sul, Brazil• I made available several new data streams for analytics and reporting making the integration between Online Transactional Processing (PostgreSQL, SQLServer, MySQL) and Online Analytical Processing (Redshift) ETL architectures using AWS services.• I created dynamic and user-friendly dashboards on Sisense for a diverse range of business types, enabling customers to easily track and analyze key performance metrics specific to their needs, supporting clients in making value from analytical solutions. -
Data AnalystHcpa - Porto Alegre Teaching Hospital Jun 2017 - Mar 2021Porto Alegre E Região, Brasil• I conceived and developed the first web databases for mucopolysaccharidoses, working as a full-stack developer, on a PHP-based application hugely based on a MySQL database, providing a comprehensive and accessible platform for researchers and medical professionals. This project led to a publication in a high-impact factor (4.2) scientific journal.• I conducted research in Machine Learning in R and Python, focusing on feature selection and classification of rare diseases. This involved leveraging algorithms and identifying key patterns and markers for accurate disease classification.• I created using R programming statistics pipelines for biological research. These pipelines streamlined data ingestion and standardization in order to be useful to automatically compare across species and NGS experiments.
Luis Soares Education Details
-
Data Management -
Bioinformatics
Frequently Asked Questions about Luis Soares
What is Luis Soares's role at the current company?
Luis Soares's current role is I make high-quality data be quickly available | Data Engineer | AWS CCP | Scala | Airflow.
What schools did Luis Soares attend?
Luis Soares attended Iu International University Of Applied Sciences, Universidade Federal Do Rio Grande Do Sul.
Not the Luis Soares you were looking for?
-
Luis Soares
Brazil -
-
Luis Soares
Lauro De Freitas, Ba -
Luis Soares
São Joaquim Da Barra, Sp
Free Chrome Extension
Find emails, phones & company data instantly
Aero Online
Your AI prospecting assistant
Select data to include:
0 records × $0.02 per record
Download 750 million emails and 100 million phone numbers
Access emails and phone numbers of over 750 million business users. Instantly download verified profiles using 20+ filters, including location, job title, company, function, and industry.
Start your free trial