- Managed end-to-end data processes, business KPI understanding, data analysis, and data source identification.- Developed insightful reports and interactive dashboards on Power BI, extracting data directly from SQL Synapse, enabling stakeholders to make data-driven decisions effectively.- Designed and implemented ETL pipelines on Azure, extracting data from diverse sources, including SAP and Excel, and loading it into Azure Data Lake and SQL Synapse data warehouse.- Utilized Python and PySpark scripting to automate the data ingestion and transformation processes, improving efficiency and reducing manual intervention.- Maintained data pipelines and performed routine monitoring to proactively identify and resolve potential issues.

- Responsible for GCP cost optimization by reducing query processing costs on Bigquery and Sprinkle.- Developed data pipelines for integrating data from Snowflake to Bigquery along with SFTP to Bigquery on GCP Composer.- Responsible for monitoring DAGs with Apache Airflow and Slack for the status alert message of a DAG task.

The research was to build an application on top of smartphone sensors that collects raw data and sends it to the server. Then server filters the data using the machine learning models and displays it on the dashboard. The filtered data would be the detected features about the road and the driver by the model i.e. nature of potholes, broken patches, speed breakers, defective street lights, self honking, others honking, quality of driving. And then model directs and rates the user based on these features.