[Bluecard Visa holder in Berlin, Germany]As a seasoned professional with 5+ years of experience in software and data engineering, I have a proven track record of delivering efficient and high-quality solutions using various cloud providers' data stacksMy key accomplishments include:* Designing and optimizing data processing pipelines to ensure scalability, reliability, and extensibility + Developing batch, near-real-time, and real-time data processing pipelines + Ensuring solution quality and reliability through: - Unit testing, logging & monitoring, data observability* Developing in-house data platforms using open-source tools, allowing for increased control over infrastructure maintenance and reduced vendor lock-in risks* Building frameworks for Data Governance initiatives, including data discovery and access managementThings I use:* Programming languages: Python, SQL, NoSQL* Data processing: Apache Spark, Apache Kafka, Apache Airflow* Databases/ Datawarehouses: Postgres, BigQuery, Redshift, SQL Server, Duckdb* Cloud platforms: Azure, GCP, AWS/ Cloud native* Containerization: Docker, Kubernetes* Version control: git* Infrastructure as Code (IaC): Terraform* Shell scripting: Bash, MakefileThings I build:* Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT)* Data warehousing with Data/ Delta Lake* Real-time data processing: Streaming, near real-time to real-timeThings I follow while building: * Object-Oriented Programming (OOPS)* SOLID development principles* KISS/ DRYThings I have used (before 2 years):* Azure Event Hubs, Azure Databricks, Azure Data Factory* Neo4j Graph Databases* On-premise systems* Azure Function Apps, Azure Cosmos DB, Logic Apps, Azure Stream Analytics* Power BI