Army Research Directorate, Mechanical Sciences Division, Vehicle Structures and Dynamics Branch6.1, 6.2 Artificial Intelligence/ Machine Learning/ Deep Learning/ Reinforcement Learning/ Robotics research

Multi-Agent Reinforcement LearningMaster's ThesisGRASP Laboratory @ PennDoD SMART Fellowship ScholarAbstract:In this report, we propose a novel deep Multi-Agent Reinforcement Learning (MARL) approach for learning cooperation in a team of agents -- where the team is tasked to solve the Active Information Acquisition problem, Multi-Agent Multi-Target Tracking (MAMTT). The goal of which requires a team of agents with on-board sensors in a partially observable state to choose sequential actions to acquire information about targets of interest. One of the main constraints in existing multi-agent planning methods is the requirement of heavy online computation in order for effective execution. In contrast, with an extended training stage, deep neural network policies, trained by reinforcement learning, are efficient to execute online. Therefore, we present the MARL algorithm, Parameter Shared Clipped Double Q-learning, that trains a policy centrally for efficient decentralized control. It is often the case in partially observable, cooperative robotic scenarios that the decentralized agents must understand that they are part of a team in order to utilize a variable amount of permuted information. With that in mind, we employ the Set Transformer model to bring attention to cooperation while also tackling the problem of restricted observation sizes in reinforcement learning. The incorporation of the Set Transformer model adds flexibility and has direct applications to scalability. In respect to the MAMTT problem, rather than learning a policy to track a fixed number of targets with a fixed sized team, we learn a flexible policy called Parameter Shared Set Policy (PSSP), which considers a permutation invariant set in the form of a variable number of targets beliefs. At execution, a copy of the trained PSSP is given to each agent, allowing for cooperative teams composed of any number of homogeneous agents.

Vehicle Technology Directorate (VTD), Mechanics DivisionAdvisors: Dr. Mulugeta Haile, Dr. Jaret Riddick6.1, 6.2 DoD research in machine/ deep learning analytics in mechanics and dynamics application (Structural Health Monitoring)

Advisor: Dr. C NatarajVillanova Center for Analytics of Dynamic Systems (VCADS)Undergraduate Senior ThesisAutomatic Predictor of Cardiovascular EventsCardiovascular and cerebrovascular events (i.e., myocardial infarction, stroke) are the leading cause of premature death and disability in developed countries. This has generated great interest in the creation of automatic prognosis and diagnostic tools to accurately identify these events. The goal of this algorithm is to aid cardiologists in diagnosing patients and reduce the time it taken to reach a decision. The current diagnostic process for patients with potential cardiovascular and cerebrovascular ailments is extensive. Cardiologists comb through the ECG data in order to observe certain tell-tale signs. These manual sources of detection have a high potential for missed diagnosis; this report will outline an alternative algorithm-based method of detection that is both faster and more accurate.

Investment Operations Teamo Utilizing Visual Basic for Applications (VBA) in Excel to automate trade settlement process and data analysiso Corresponded with brokers and custodians to process and settle tradeso Executed SQL statements to update front end user-interfaces

Advisor: Dr. C NatarajDiagnostics of Fault Bearing Systemo Fall of 2016 will implement code onto a Raspberry Pi in order to create a mobile, real-time fault bearing system identifier producto Successfully wrote data-based diagnostic code to train and test Matlab’s artificial neural network to identify specific faults in a bearing system (95%+ success rate)Abstract:The purpose of this research is to train an artificial neural network in order to identify and diagnose defects in a bearing system through vibrations of the shaft in a data driven based framework. There are five sets of data that are collected from the bearing system including the outer race defect, inner race defect, ball defect and the combination of outer race and inner race defects. Through the use of proximity sensors and vibration analysis, the characteristics of the bearing system can yield information regarding the machine’s condition. Five trials of ten sets of speeds from 5-50 Hz of each of the five situations produces 250 sets of data. From the measured data, statistical features in the time domain are extracted and fed to an artificial neural network. The trained neural network is then used for prediction of the faults in the bearing using the test data. Ball pass frequencies are also calculated and inspected through the Fourier power spectrum in order to validate the prediction. The research will give insight into the workings of data acquisition systems, data analysis, and diagnostics.

• International boat competition where student teams race autonomous surface vehicles of their own design through an aquatic obstacle course. Competition tasks include navigation and control, obstacle detection and avoidance, symbol recognition and vessel docking, underwater beacon detection, and interoperability• AUVSI Roboboat competition- 5th overall static judging (June 2015 result)• Systems Integration Engineer- Ensured compatibility of subsystem components in an integrated working system• Manufacturing Engineer- Introduced water-proof box used in RobotX onto a new set of fiberglass pontoon hulls and frame fit for Roboboat Competition• Vision System Engineer- Developed software to identify color and shape through the use of on-board cameras and LIDARFunded by:Naval Research Enterprise Internship Program (NREIP)Office of Naval Research (ONR)