In response to NASA STTR topic T4.01, Information Technologies for Intelligent and Adaptive Space Robotics, Alphacore Inc. in partnership with the Arizona State University (ASU) School of Earth and Space Exploration will develop a low-SWaP-C, high-performance, extreme perception and vision system for autonomous robot operations. Our novel radiation-hard adaptive dual-mode neuromorphic (event-based) vision system configuration is designed to provide terrestrial autonomous robot-comparable 3D object detection, depth estimation, mapping, and tracking functionality for future use on lunar and planetary surfaces.
Our approach is to recognize that ultra-high performance terrestrial state-of-the-art image processing microelectronics hardware likely will not be available. To address this constraint, our solution is to instead selectively reduce or throttle the image data from the image sensors to the downstream image processing microelectronic hardware. By selectively (and significantly) reducing the image data rate from the camera(s), lower performance space-qualifiable image processing electronics will then be able to provide the needed functionality on the now much sparser information from image data. The challenge is reducing the image data rate, while still providing the terrestrial-comparable state (pose and velocity) estimation, 3D object detection, depth estimation, mapping, and tracking functionality needed for autonomous operations.
We propose to solve this challenge by replacing existing conventional space-grade CMOS frame-based cameras with a novel radiation-hardened version of a Dynamic and Active Pixel Vision Sensor (DAVIS) dual-mode image sensor , which combines a conventional global-shutter CMOS camera with an event-based image sensor (EBS) in the same pixel array. In practice, event-based image sensors can reduce the downstream computational burden by an estimated one to two orders of magnitude.
Future NASA missions that could benefit from these advancements include the Lunar Gateway, expected to be unmanned for 92% of the time and thus, rely heavily on robotic systems for maintenance and repair, and a planned ‘tunnel-bot’ to pierce through the icy surface on Europa to study the space underneath. It can also help the Cold Operable Lunar Deployable Arm (COLDArm) project, one of NASA’s technologies being developed to enable future missions to extreme environments on the Moon, Mars, and ocean worlds such as Jupiter and Saturn moons.
Alphacore’s solution can be applied for use in maintenance and repair of defense and commercial space systems. Alphacore’s radiation-hardened perception technology can help add autonomous capabilities to defense unmanned aerial vehicles and autonomous terrestrial platforms, such as future robotic combat vehicles and optionally manned fighting vehicles.