A quick learner, creative thinker, and adept in Python, PySpark, ETL techniques, SQL/noSQL, Git, and GitHub. With over 6 years of professional experience, I have made significant contributions in open-source projects like Shikshalokam in developing & optimising robust data plumbing systems and ETL processes. Exceptional communication skills with experience in clients such as Udaan and HSBC. Rapid learner adept at collaborating on evolving technologies. Committed to helping companies create efficient and scale-able data pipelines around big data to integrate machine learning algorithms.- Development:Python, PySpark, BASH, SQL, Scala (Beginner)- Technologies:Hadoop, Apache Kafka, Apache Druid, Apache Superset, Apache Flink, Prefect, Terraform, Docker, Fast API, Flask, Selenium- Cloud Products:AWS: IAM, Cloud shell, Lambda, EC2, S3, RedshiftAZURE: Azure StorageGCP: IAM, Cloud shell, Big Query, Cloud Storage, VMs, Data ProcOracle: IAM, Networking, Compute, Storage, Security - Database:Mongo DB, Cassandra DB, PostgreSQL- Others:Databricks Foundations
-
Senior Data EngineerBlack PianoWest Bengal, India -
Senior Data EngineerBlack Piano May 2024 - PresentIndia -
Data EngineerCloudcraftz Solutions May 2022 - Apr 2024India1. Shikshalokam- Maintained data pipeline up-time of 97.8% while ingesting streaming and transactional data across 8 different data sources using PySpark, Druid, Azure Storage, and Python.- Orchestrated the data migration from Apache Druid to CassandraDB, transitioning to event-based data capture increasing performance by 17%.- Utilised Apache Flink, and Kafka for real-time event pre-processing, enabling dynamic Cassandra-based reporting to be used in BI tools like Apache Superset.- Developed cloud-agnostic data storage architecture, automated data-pipeline logs, and proposed MongoDB schema improvements for enhanced user data validation for support team usability.2. HSBC- Developed an internal dashboard using ETL and PySpark for data processing from global sources,stored in BigQuery, and visualized with Looker, reducing manual workload by 34%.- Enhanced business by organizing exact geo-locations from user addresses during onboarding and cardissuance, focusing on credit card-only users.- Executed SQL scripts using gbq CLI commands to supply pertinent data to stakeholders for spontaneous inquiries.3. Udaan- Analyzed transactional data to identify Profit & Loss trends for specific commodities. Developed a predictive time-series model, leading to a 28% improvement in vendor procurement KPIs.- Scraped data from reference websites detailing daily commodities prices to facilitate price prediction for a day and a week to ensure inventory model as well as JIT model.- Designed the database schema for vendor deals allowing for a systemic commodity procurement based on price, quantity and vendor relationship. -
Technical Executive - Data ServicesConcentrix Jul 2019 - May 2021Bengaluru, Karnataka, India- Handled complicated data issues by restoring damaged corporate files for Quick-books Enterprise &Premier products utilizing MySQL commands (foreign keys, inner joins) with a satisfaction score of89.3%.- Utilized Python, Docker and PowerBI to extract and load small business data or broken databases andre-load in a MySQL database for usability for end-customers. Reports were shared to stakeholders forgaining insights on issues related to the software.- Capable of handling system-related issues, file-related issues, and small business-related accounting & bookkeeping issues. -
Customer OperationsAmazon Aug 2016 - Apr 2019Hyderabad, Telangana, India- Managed customer relations, participated in the pilot team for Concession Abuse Prevention, helpingcreate business protocols and policies based on live customer queries.- Served as a Subject Matter Expert (SME) in helping new hires understand Concession Abuse, along withhelping team managers with sales queries data collection and analysis allowing for a streamline processfor better business-oriented results.- Monitoring and assigning tickets regarding payment failures andcheque cancellations along with handling professional mail escalations decreasing the payment failures by 15%.
Snehangsu De Education Details
-
Electrical And Electronics Engineering -
Margaret (S.N) English SchoolScience
Frequently Asked Questions about Snehangsu De
What company does Snehangsu De work for?
Snehangsu De works for Black Piano
What is Snehangsu De's role at the current company?
Snehangsu De's current role is Senior Data Engineer.
What schools did Snehangsu De attend?
Snehangsu De attended Jis University, Margaret (S.n) English School.
Not the Snehangsu De you were looking for?
-
3yahoo.com, lntinfotech.com, citigroup.com
-
Snehangsu De
India1gmail.com
Free Chrome Extension
Find emails, phones & company data instantly
Aero Online
Your AI prospecting assistant
Select data to include:
0 records × $0.02 per record
Download 750 million emails and 100 million phone numbers
Access emails and phone numbers of over 750 million business users. Instantly download verified profiles using 20+ filters, including location, job title, company, function, and industry.
Start your free trial