Data Engineer Ii
CurrentData aNd AI team (DNA)Main Languages for Development: Python and SQLFrameworks: Spark, Medallion Arch.Libraries: PySpark, Pandas, azure.storage.filedatalake, PyArrow, matplotlib, NumPy, sklearnAccomplishments in the DataBricks platform:- Engineered pipelines from scratch on application statistics sourced from MSFT Graph and Github apis.- Built out JIRA analytics that provide OpEx & CapEx expenditures, Ticket status and hourly analysis to NCR Voyix's board of directors.- Advanced United States Postal Services' dashboard by shifting their processes to utilize Azure Data Lake Stroage and Databricks workflow automations. Time saved customer ~2 hours daily from slow refreshes and prevented future memory problems in PBI.- Constructed a python notebook for Tableau usage statistics and monitoring. Saved DevOps Engineers about 3 hours of work of pulling and cleaning data for each data pull.Current Projects:- R&D on chat-bots leveraging Azure Open-AI.- R&D Retail, exploring LLM capabilities to help build customer's shopping list based on requests, feature, recipe, etc.- R&D Restaurant, Customer Personalization based customer's purchasing history (e.g. giving customer discount on a less busy day or time for their favorite item).- building out Azure ML space for HR Analytics projects.- finalizing dashboard for NCR Voyix's board of directors.- Implementing USPS's API to make the data more streamlined.- Maintaining HR Analytics platform and expertise.