- Led squad's effort to migrate the catalog format of Toast's data lake from AWS LakeFormation to Apache Iceberg

- Implemented a deletion queuing system for the safe deletion of older generations of streaming data- Created a custom data backup system that allowed us to backup tables from our data catalog, as well as store metadata which would allow us to re-register a backup as a snapshot table in our catalog- Helped migrate our stack from using Spark 3.2 to Spark 3.4- Worked on implementation of cost tagging initiative to evaluate and reduce team's spend on data storage and computation-Implemented Auth0 across our team's resources

- Continuing to implement new ETL jobs from various data sources such as SQL Server, S3, client side SFTP etc using clean, testable Python code.- Mentoring junior engineers- Working with R&D to implement highly performant Spark jobs in Scala- Converting legacy SQL code into Python for better performance and Source Code control- Improving unit test coverage on existing pipelines- Implemented time intensive Athena calculations and data load in Scala to save engineer hours and reduce human error- Designed and built pyspark job runner to automate submission of pyspark jobs to EMR including handling of dependencies and job configurations- Continue to maintain existing Python code base, expanding test coverage, improving performance and scalability- Rewriting legacy Python code in Spark to improve performance and reduce need for constantly running EC2 instances.

- Implemented dozens of new ETL jobs in SQL and Python using both T-SQL and Postgres- Ensured data cleanliness in all 4 of our SQL Server databases as well as our S3 data storage- Ran, maintained and improved a monthly and quarterly Spark job written in Scala- Implemented On-Call/Pipeline Triage system for engineers- Assisted with migration from On Premise/Bare Metal SQL Servers to AWS RDS Servers in the cloud- Assisted with migration of job scheduling from a combination of Cron and Cloudwatch Events to Airflow

Lloyds Banking Group, remediation project, analytics. My involvement includes:- Interrogation and analysis of very large volumes of data from different sources in order to understand business logic used from each source- Produce powerpoints detailing business logic to be applied for key business stakeholders- Receive, analyse and prepare data to be loaded into the system database- Produce detailed reports for key business stakeholders which 'tell the story' of the data in order to understand appropriate treatmentQBE Insurance, implementing the Solvency II Pillar 3 Reporting Solution. My involvement included end to end delivery of certain aspects including:- Analysis and interrogation of large volumes of data in order to gain insights into the functionality of the solution- Business Analysis, Design and prototyping of Analytics solutions in order to analyse and identify issues within data- Design and build of Exceptions Reporting solution in SQL- Key contributor to analysis, design and test of complex components of Financial Reporting solution built in SQL- Writing test approach, test model and executing several batches of tests for the Technical Provisions Model.- Analysing, executing and writing Microsoft SQL scripts- Performing defect triage for team of 5 testers- Running client meetings in order to identify and gather requirements- Communicating solutions to key stakeholders

Summer Placement with Asset Management team.Conducting visual inspections of parts of coal fired power plant during outage.Drafting documents detailing findings of visual inspections and recommending maintenance.Drafting legally required document summarising findings throughout whole outage and maintenance carried out.Carrying out design calculations for new equipment.Collecting and compiling data comprehensively for use in designing schemes of examination for pipework.

Finding and analysing customer bills to determine whether correct or not.Sending customers revised bills or credit notes depending on my findings.