Associate Data Engineer
Current๐๐ง๐ ๐ฃ๐ถ๐ฝ๐ฒ๐น๐ถ๐ป๐ฒ ๐ ๐ฎ๐ป๐ฎ๐ด๐ฒ๐บ๐ฒ๐ป๐: Own and implement ETL pipelines using Azure Data Factory, ensuring seamless data ingestion and transformation processes that are critical to our operations.๐ฅ๐ฒ๐ฎ๐น-๐ง๐ถ๐บ๐ฒ ๐๐ฎ๐๐ฎ ๐๐ป๐ด๐ฒ๐๐๐ถ๐ผ๐ป: Utilizing Azure Databricks and PySpark to facilitate real-time data ingestion, enabling timely and accurate analytics for our stakeholders.๐๐ฎ๐๐ฎ๐ฏ๐ฎ๐๐ฒ ๐๐ฑ๐บ๐ถ๐ป๐ถ๐๐๐ฟ๐ฎ๐๐ถ๐ผ๐ป: Manage and optimize our SQL database, ensuring that it meets the performance and reliability standards required for our applications.๐๐๐๐๐ผ๐บ๐ฒ๐ฟ ๐๐ป๐ด๐ฎ๐ด๐ฒ๐บ๐ฒ๐ป๐: Work closely with customers, translating their technical requirements into actionable data solutions, and addressing any specific requests related to data formats.๐๐๐๐๐ผ๐บ๐ฒ๐ฟ ๐ข๐ป๐ฏ๐ผ๐ฎ๐ฟ๐ฑ๐ถ๐ป๐ด: Lead the end-to-end customer onboarding process, from setting up data pipelines to ensuring smooth transitions from UAT to production.๐๐ฎ๐๐ฎ๐ฏ๐ฎ๐๐ฒ ๐ ๐ถ๐ด๐ฟ๐ฎ๐๐ถ๐ผ๐ป: Actively involved in migrating databases from SQL DB to Azure Cosmos DB, including designing pipelines to integrate Cosmos DB as a sink.๐๐ถ๐ด๐ต-๐๐ฟ๐ฒ๐พ๐๐ฒ๐ป๐ฐ๐ ๐๐ฎ๐๐ฎ ๐ฃ๐ถ๐ฝ๐ฒ๐น๐ถ๐ป๐ฒ๐: Have developed specialized pipelines for Carbon Capture projects, which handle high-frequency data and perform averaging at various time intervals to meet project needs.๐๐ฎ๐ฐ๐ธ๐๐ฒ๐๐๐ถ๐ป๐ด & ๐ ๐ผ๐ฑ๐ฒ๐น ๐ง๐๐ป๐ถ๐ป๐ด: Responsible for performing backtesting on historical data for our customers and fine-tune the data science model through hyperparameter tuning, particularly for chemicals industry customers.