Data Engineer
CurrentTook part as data engineer in multiple projects to create predictable and reliable Data Warehouses.Projects included industry standard technologies such as cloud and on-premis solutions.Responsibilities:- Streaming processing with Apache Spark, Kafka, Kafka Streams and Kafka Connect (Debezium)- Batch pipelines from/into HDFS, S3, Azure DataLake to OLAP systems (Snowflake, Redshift)- Development of Data Marts with dbt- Development and support of Kubernetes deployments- Ad-hoc analysis with Splunk- Infrastructure maintenance (provisioning with Terraform, manual optimizations), troubleshooting, customers supportAchievements:- Optimized cloud infrastructure that resulted in 40% cost reduction- Cold storage optimization that reduced occupied space by 60%- Created a dev environment for the team- MentorshipSkills: Snowflake, AWS Redshift, AWS Firehose, Scala, Apache Spark, Kubernetes, HDFS