Evaluated a learning algorithm using perturbation and explicit character-level representation to improve learning robustness.Evaluated a BERT model on a sentiment analysis task.Incorporated ELMo embeddings to explicitly incorporate character level representations.Characters were realistically perturbed at training time.Found a strong benefit to perturbing characters and a limited benefit to augmented embeddings.

Replicated the findings of “Overcoming Catastrophic forgetting in neural networks” by Kirkpatrick et al., “Learning without Forgetting” by Li et al., and “Continual Learning Through Synaptic Intelligence” by Zenke et al.Explored the possibility of combining techniques to produce even more robust models.

Developed in Python with PyTorch.Explores the use of a penalized loss function to learn convolution-like structures by penalizing the distance between neurons.