Senior Data Engineer/ Database Developer
CurrentI excel in end-to-end data processing and analytics, specializing in real-time data handling with Kafka, Spark, and Scala. My expertise includes designing ETL pipelines, optimizing Snowflake resources, and enhancing RDBMS query performance. Data security is a top priority, with measures like RBAC and encryption. I architect cloud-based data solutions on AWS, using services like S3, Glue, Redshift, and EMR, and streamline ETL workflows with AWS Glue. My skill set covers scaling Snowflake warehouses, Python-based data ingestion, data cleansing, and SQL-driven workflows. Proficiency in Hive, CI/CD orchestration, PostgreSQL data migration, and Tableau visualization further enhances my capabilities.My professional journey demonstrates my proficiency in managing data migration projects involving PostgreSQL databases. I excel in data extraction, transformation, and loading (ETL) tasks, ensuring seamless data transfer. Collaboration is central to my approach, as I work closely with business stakeholders to automate processes and create impactful data storytelling visualizations in Tableau. I also leverage Databricks Delta Lake, enabling the creation of versioned Apache Parquet files with transaction logs and audit history for data reliability and traceability. To optimize data processing, I employ SSIS parallelism and multithreading, resulting in improved performance and reduced ETL durations. Crafting visually appealing dashboards on Power BI to provide valuable insights for data-driven decision-making is another strength.Furthermore, I've honed my skills in working effectively with the Cloudera distribution on AWS EC2 Instances, specializing in Spark for performance optimization. My commitment to efficient code management is evident through my extensive use of version control tools like Stash, Bit-Bucket, and GitHub.