Architected and developed Momentum, a real-time streaming data application using PyFlink, Kafka, AWS Kinesis, and DynamoDB for handling and processing high-volume streams.Developed streaming solutions reading from Kafka topics, ingesting data into Kinesis Streams, and providing real-time insights for the backend web applications.Designed and managed data pipelines processing terabytes of data daily using AWS Glue with PySpark and AWS Lambda functions, ensuring smooth data ingestion and batch processing.Migrated on-premises databases to AWS Data Lake with AWS DMS and S3, ensuring reliable and scalable storage for further analysis.Built a loyalty program using Django and FastAPI, fully integrated into our core business platform, driving increased customer engagement and retention.Implemented AWS Lambda functions for handling API calls to PowerBI and other third-party services to aggregate data and store them in S3 and Redshift.Maintained monitoring and alerting using AWS CloudWatch for real-time tracking of system performance and data quality.Managed batch and streaming pipelines for large-scale data flows, optimizing performance and reducing downtime.

www.meduzza.ai

Design and upkeep of database warehouse utilizing GCP BigQuery and GCS.Implementation of ETL processes involving SQL, Python, Groovy, Apache Nifi, and Apache Airflow.Creation of minor web services using flask-appbuilder.Creation of dashboards for exploring and visualizing data using Apache Superset.Engagement in database modeling and design, performance optimization, development of stored procedures, and execution of ad-hoc queries, employing PostgreSQL and MySQL.Management of code using GitLab and Git.Designing Apache NiFi flows to automate the retrieval of client tokens, initiate API requests for publisher data, and seamlessly transform and store the acquired information into a database.Wrote a couple of smart contracts on Ethereum Blockchain using Solidity

- Designed, developed, and maintained ETL pipelines using Python Pandas, PySpark, and SQL for data processing and transformation.- Leveraged Kimball Approach for data modeling and designed database schema using PostgreSQL, AWS, and Snowflake.- Utilized Docker for containerization and CI/CD for automated deployment and testing.- Rebuilt integration from DataIku to Airflow, enhancing the efficiency and reducing the processing time significantly.- Collaborated with cross-functional teams to identify data needs, ensure data quality, and optimize data processing workflows.

Developed Python-based applications utilizing Django and Flask for internal projects, improving the efficiency of data processing and management workflows.Designed a mini application to process large datasets by implementing PySpark, which distributed data efficiently and optimized performance for handling large-scale data ingestion and processing.Created API integrations with internal systems, automating and streamlining the communication between various services and improving data consistency across platforms.Collaborated with the analytics team to provide scalable solutions for data transformations and reporting.Played a pivotal role in the development of robust data governance policies and procedures, ensuring compliance with industry standards and regulations.Used ETL processes by utilizing Amazon Web Services (AWS), Apache Spark, Python, and SQL to optimize data extraction, transformation, and loading processes, contributing to the seamless flow of high-quality data across the organization.

Developed and maintained automated test scripts using Python and Selenium, ensuring the functionality and performance of web applications.Collaborated with development and QA teams to understand product functionality, and created comprehensive test plans to address key testing scenarios.Executed automated test cases, identified bugs, and worked closely with developers to troubleshoot and resolve issues promptly.Monitored and reported test results, tracked defects, and contributed to maintaining overall application stability.Assisted in designing and implementing automated frameworks, improving testing efficiency and enhancing the repeatability of tests.Performed manual testing to complement automated efforts, ensuring comprehensive coverage of all application features.Kept up-to-date with the latest automation tools, trends, and best practices to continuously improve testing processes.