Led the development and upkeep of Autoloader and Streaming jobs on Databricks, utilizing PySpark for efficient data processing. Specialized in Structured Streaming for real-time data needs, employing Kafka for reliable data streaming and Apache Airflow for optimal job scheduling. Proficient in troubleshooting, ensuring uninterrupted data flow by addressing errors in data jobs. Utilized AWS Glue for extracting, transforming, and loading data, implementing Lambda functions for automated data processing tasks. Developed Python scripts for file processing, ensuring compliance with downstream requirements. Conducted performance optimization of data tasks, guaranteeing seamless integration and data integrity across systems

Spearheaded support for faculty members by managing project work, facilitating classroom instruction, overseeing exams, and maintaining accurate record-keeping, ensuring smooth academic operations.Tutored students in Web designing, communicated via Blackboard, and collaborated during office hours for academic success. Assisted in refining course plans, graded exam papers, helped students in troubleshooting issues with operating systems and installing the applications required for course work.