Data Engineer
Fraud Detection (Neo4j)-Designed a tailored graph model that suits the fraud detection use case.-Configured Neo4j on remote instances, optimizing performance and enhancing security.-Built an incremental data pipeline that is running daily to load the base data from DataWorks into Neo4j using Pyspark.Reshaping and Reorganizing Cloud Data Warehouse -Implemented a standardized naming convention for tables in the cloud data warehouse.-Developed and documented Standard Operating Procedures (SOP) for naming conventions to ensure consistency and ease of dataretrieval.-Collaborated with cross-functional teams to plan and execute seamless table migrations.-Improved overall system performance and streamlined data access by strategically relocating tables. Reconciliation and Settlement Report-Developed SQL queries for ingesting data from diverse sources such as Kafka and MySQL, aligning with QA requirements.-Created shell scripts to export data to Alibaba Cloud OSS, ensuring seamless data flow.-Scheduled and automated the data pipeline, enhancing efficiency with regular execution and error handling mechanisms. Data Governance & Security-Designed and developed an interactive dashboard to monitor the company’s employees for not having excessive permissions on DataWorks thus preventing data leakage by not allowing them to query sensitive tables.-Implemented Label-based Access Control (LBAC) on Alibaba Cloud to enhance the security of sensitive data within the company's databases.-Configured and applied labels to specific columns containing sensitive information, ensuring a fine-grained access control mechanism.