(MATS extension program) Continuing my work on targeted LLM deception and manipulation. Paper: https://arxiv.org/abs/2411.02306

Wrote the paper "Targeted Manipulation and Deception Emerge when Optimizing LLMs for User Feedback" which looks at how deceptive and manipulative LLM behavior can arise from imperfect feedback, particularly user feedback.

Long-Term Future Fund funded project which showed switching to a multi-objective reward function in the preference model of a RL from AI feedback system improves performance. Paper: https://arxiv.org/abs/2406.07295

Facilitating multiple in person groups for the AI Safety Fundamentals Alignment course

Working on comparing the expressivities of different Reinforcement Learning formalisms. Traditional Markov rewards cannot express objectives like risk aversion, max-min or lexicographic priorities which could be important for training safe agents. This work investigates other formalisms such as Multi-Objective RL, Reward Machines, Linear Temporal Logic and Convex RL among many others, to see which of these can express all possible tasks expressible by another formalism.

I worked on creating a system to detect, classify and segment bone marrow fragments in blood. This included automating the focusing and image acquisition process from bone marrow aspirate smears and training several vision transformers and CNNs.

I developed a program that visualized how users interact with Axis' mobile applications, enhancing our understanding of user behavior and leading to improvements in the overall user experience. I also analyzed error spikes and abnormal start-up times or latencies, contributing to the improvement of our mobile applications. I gained experience working with SQL databases, BigQuery, and Looker Studio.