Created a data pipeline by extracting, cleaning, normalizing, and enriching data.- Worked as a Senior Data Engineer in a team of 11- Developed, tested, deployed, and maintained the end-to-end data pipeline- Built a risk dashboard in Redash to visually detect anomalies- Utilized Python, Spark, and SQL for implementation- Tested the pipeline on EMR notebooks- Conducted orchestration testing using Airflow- Merged code in production using GitHubTechnology Involved : Pyspark, Python, SQL, Pycharm, EMR, AWS, Git, Github, JIRA, Confluence

Developed cost-effective and accurate data pipeline approach using various AWS cloud services- Worked as a Data Engineer in a team of 12- Worked on multiple data sources like Kafka, dynamodb, rds, JSON, text, csv, etc- Applied optimizations and space reduction on data, code, and pipelines- Built scalable and generic etl processes, ensuring data integrity and pipeline stability- Created documentation and analytic reports, delivering summarized results to stakeholders- Designed database architectural strategies for various projects- Collaborated with cross-functional teams for efficient development- Reduced S3 cost by 50% using vacuum and optimization- Decreased dms process task cost by 30% by splitting instances across different azsTechnology Involved :Pyspark, Python, SQL, Databricks, AWS ( DMS, RDS, Lambda, SQS, SNS, EC2, Glue ), Metabase, Presto

Implemented, designed, developed, and unit tested the invoice handling/validation pipeline for daily invoice rejected reports- Worked as a Specialist Programmer in a team of 21- Configured pipeline using Azure databricks and Apache airflow for data processing and orchestration, and Azure DevOps for deployment- Optimized, purged, and vacuumed delta tables, reducing cost by 40%- Maintained system's highest standards with frugality- Changed framework to use delta tables, reducing blob archival cost and improving query speed by 600ms on average- Delivered services within specific timeframes by managing 3 pipelines simultaneously- Built data flow to handle orders from REST API using nifi and CRUD operations- Utilized incorta bi to analyze ITSM dashboards and identify areas for improvement in client storesTechnology Involved : Pyspark, Python, Databricks, SQL, Azure, Apache NiFi, Incorta BI, JIRA, Postman

Tested the quality of the telecom products such as PBX, Gateways, VoIP phones, SLT phones. Documented the test cases as per the user stories/ SRS, reviewed them, tested the features, network and interface as per the testcases. Worked on agile environment with spring breakdown to 1 week cycle with daily standup meetings and retro at the end of each sprint. Done unit testing, integrated testing, regression testing, load testing, monkey testing, negative testing.