Description: Image Recognition with Magic Leap using 5G capabilities for improved performance. Magic Leap is used to capture frames and send them to a python machine learning server, which in turn will process and send the results back to Magic Leap. Then the device will represent these results on the original image that is taken to show the user the processed image. AR devices can benefit from offloading processes to other devices, such as a nearby server, since offloading computation tasks on servers can save on power consumption and heat dissipation. Due to the valuable application of object recognition, the team focuses on offloading these tasks from Magic Leap headsets to a nearby edge server. The team architected a Framework for Enhancing Augmented Reality Using Mobile Edge Computing (FEARMEC) to handle the offloading, integration of artificial intelligence (AI) and AR, and network trafficking. Briefly, the framework takes in the video feed from the AR device, selects keyframes (i.e., screenshot) from the video feed and sends them to the edge server. The server executes an object recognition AI and returns the result to the framework, which then overlays the information on the user’s headset point of view. The project involved using ML1 headset, NVIDIA Jetson, and NetGear Nighthawk (5G connectivity). The software development included C# coding for ML1 headset, Python for the edge server, and socket programming for the network traffic control/communication.
Team: Primary Investigator: Hammam Alsafrjalani Developer: Berk Basarer
