Data Engineer
Current- Streamlined data collection and transformation, ETL, using Talend Open Studio, ensuring data aligns with STC business requirements.- Secured data integrity and accessibility by loading transformed data into a PostgreSQL database.- Enhanced data insights, monitoring and decision-making capabilities for top management through data visualizations.- Gained knowledge in Linux command-line tools, process management, and system administration by deploying frontend and backend code to production servers.- Developed and implemented data routing and system logic using Apache NiFi.- Established a high-performance messaging infrastructure capable of processing millions of Kafka messages per minute.- Designed and architected a resilient and scalable data flow system that efficiently handled real-time data ingestion and processing.- Leveraged Apache NiFi's visual interface to create and manage complex data flow graphs with ease.