Senior Data Engineer
CurrentResponsibilities: • Implementing data storage and processing solutions in the Google Cloud. • Designing, implementing, and maintaining efficient and scalable data pipelines. • Knowledge of orchestration tools such as Apache Airflow to schedule and monitor workflows. • Implementing data governance best practices to ensure data security and privacy. • Performing raw data cleansing and transformation using SQL and Python. • Managing data storage and processing in BigQuery and Google Cloud Storage. • Integrating data from multiple sources, ensuring data consistency, integrity, and quality. • Data Modeling: Solid understanding of data modeling concepts and experience in implementing efficient data schema. • Optimizing performance by performing performance tuning on data pipelines, identifying and resolving query bottlenecks. • Implementing security practices, ensuring compliance with data protection regulations and security policies. • Documentation: Maintaining detailed technical documentation of pipelines, data processes, and lessons learned. • Ability to write complex SQL queries to manipulate and extract data. • Using version control systems, such as Git, for code control and collaboration. • Ability to analyze complex problems and find efficient solutions at scale. • Good communication skills to collaborate effectively with other teams.Key Achievements: • Acting in large projects in the role of data engineer, dealing directly with the construction of the DataOps Framework. • Defining practices and tools that optimize the flow of data life. • Delivery of curated and optimized data for queries and visualizations. • Cost reduction by using data optimization techniques like partition probing and partition pruning. • Knowledge sharing through articles produced via internal Wiki. • Engagement between different areas, translating requirements into real value for the business. • Achieving greater reliability and more assertive insights for users.
