Indonesia XL Data Ocean project:- Participating into Indonesia XL Axiata project in telecom industry, both on-site (4 months) and remotely, focusing on data migration to Huawei Cloud and creating ETL pipelines.- Developing SQL scripts to migrate data from different sources and between layers in database; applying business logics into fields.- Creating table migration and file distribution SQL scripts using Huawei Cloud components DataArts Studio and DLI (Data Lake Insight)- Updating migration scripts during the UAT phase based on findings, issues, and change requests from the customer.- Supporting Turkey team and HQ colleagues during the development phase.Spark Structured Streaming + Kafka Demo:- Prepared a comprehensive demo for a customer in KSA using Huawei Cloud components, including OBS (Object Storage Service), MRS (Map Reduce Service) Spark, MRS Kafka, MRS Hive, CDM (Cloud Data Migration), RDS (mySQL), and DataArts Studio.- Creating streaming flows via Python and Spark Structured Streaming with Kafka integration, writing data into Hive using Spark, and migrating data from Hive to RDS (mySQL) via CDM.- Creating DataArts jobs for both streaming and batch processing flows.- Providing technical support to the team during demos and training sessions.Malaysia - Customer Demo:- Generating JSON sample data and write into Kafka topic using Flink Java.- Reading data from Kafka and process with Flink Java, migrate into Hudi.- Migrating data from Kafka into Hbase using Flink Java.

- Extracting data from Oracle, SQL Server databases and saving as Parquet and ORC files using Spark and creating Hive external tables from files.- Scheduling migration jobs on Apache Airflow. - Extracting data using Spark Structured Streaming via Kafka Integration, comparing consumed data from topic with MapRDB Database and make calculations on it.- Implementing delta lake concept using HPE Ezmeral Data Fabric components and setting up configuration files. Changing Metastore DB of Hive, Hue and Airflow into MySQL.- Installing and configuring components of HPE Ezmeral Data Fabric.- Creating tables and volumes for data migration.- Configuring Delta - Hive connector.- Data migration from different sources (Web services, JSON and Excel files) and SQL Server database to in SQL Server using Talend workflows and database objects.- Creating ETL jobs to migrate data from Oracle and SQL Server databases to SQL Server using SSIS workflows and database objects.- Creating processes such as increasing data quality, deduplication, association, enrichment using Data Quality Services and during SSIS workflows.- Creating models, entities and attributes on Master Data Services.

- Development of Power BI models and reports, preparation of DAX queries.- Analysing Oracle database.- Responsible for ETL operational support and fixing bugs on ETL jobs using IBM DataStage.- Creating dashboards and reports using various data visualization techniques on IBM Cognos.