Airflow Migration and OptimizationMigrated data pipelines from standard Airflow toAstronomer Airflow, enhancing scalability & flexibility.Integrated with GitHub for seamless CI/CD, code changes automatically update backend & deploy DAGs via CI/CD looper.Improved monitoring and management with advanced logging and alerting features.Simplified infrastructure management, reducing operational burden and focusing on development.Enhanced deployment efficiency and reliability… Show more Airflow Migration and OptimizationMigrated data pipelines from standard Airflow toAstronomer Airflow, enhancing scalability & flexibility.Integrated with GitHub for seamless CI/CD, code changes automatically update backend & deploy DAGs via CI/CD looper.Improved monitoring and management with advanced logging and alerting features.Simplified infrastructure management, reducing operational burden and focusing on development.Enhanced deployment efficiency and reliability, ensuring rapid and consistent updates.Tech/Tools – Airflow, Github, GCS, Looper, VS Code.Azure Data Ingestion and Analysis PipelineDesigned & executed data ingestion pipelines to seamlessly transfer unprocessed data from sources (Blob Storage, Rest API, SQL Server) into Azure environment.Utilized Azure Data Factory to automate preprocessing & cleaning of raw data, ensuring quality & accuracy while reducing manual efforts.Leveraged aggregate functions in Databricks to summarize and consolidate data, enabling meaningful insights and statistical analysis.Tech/Tools – MS Azure, Databricks, Data Factory, Azure Blob Storage, ADLS gen2, Pyspark.Python-Driven PowerPoint Automation Automated monthly data refresh on PowerPoint deck using the pptx library. Automating 15 slides reduced slide update time from 15 hours (Manually) to 1 minute (Python automated). Eliminated human errors and ensured accuracy by automating the population of numerous values. Enhanced the robustness and reliability of the entire process, making it error-free. Show less

Improving existing process performance andfunctionality and fixing their performance issues.Primarily responsible for scheduling, monitoring &maintaining jobs - Daily, Weekly, Monthly and YearlyInteracting with the End users on a regular basis forresolving issues.Python, PowerBI, Data Structures, numpy, pandas,Sreamlit, Web Scrapping.

Gathered real time data collected by different users(voters) from diff. regions of Delhi by creating onlinespreadsheet for Delhi Election.Responsible for Data Cleaning, Analysis, DataVisualization and submission of reports and Dashboard.