Research Project

Design of Smart Sensor Pallet for Drones

Motivation/Research Problem
To support NASA missions, NASA scientists and researchers need to test and deploy their newly-developed algorithms and sensors on unmanned aerial vehicles (UAVs). Low-cost commercial UAVs are usually closed systems, where users cannot modify nor use the sensor data for testing or verifying algorithms. To close this gap, Dr. Geng collaborates with Mr. Mcwherter from NASA Armstrong to develop a modular and reconfigurable pallet. The pallet can be adjusted to fit onto common form-factor drones and be customized with different sensors. Also, the pallet will work as a standalone system with sensor data acquisition and processing capability.
Research Team
Faculty:

Xiaojun (Ashley) Geng – Electrical and Computer Engineering

Collaborator – September 2019 — May 2020:

Shaun Mcwherter: primary NASA Armstrong contact, and directly involved in the work

Student Team – September 2019 — May 2020:

  • Ridge Tejuco
  • Mario Chang
  • Marian Giron
  • Yasmine Goudjil
  • Oscar Tejeda
  • Alejandro Carmona
  • Mohammad Hossain

Student Team – May — August 2020

  • Oscar Tejeda
  • Alejandro Carmona
  • Stephen Oonk
  • Francisco Maldonado

Student Team – September-December 2020

  • Stephen Oonk
  • Francisco Maldonado
  • Alejandro Carmona
  • Mohammad Hossain
  • Caleb Amposta
  • Kevin Erazo
  • Robert Javier
  • Khanh Tran
  • Haley Nanas
  • Thomas Decker
  • Haroutun Haroutunian
  • Flavio Mendez
  • Nathan Speer

Collaborators January-May 2021:

John Lenko: Collaborator from Xtensor Systems Inc., and directly involved in the work
Kurt Vanlaar: Collaborator from Xtensor Systems Inc. and directly involved in the work

Student Team  January-May 2021:

  • Mohammad Hossain
  • Najmeh Najmi
  • Ilker Loza
  • Kevin Erazo
  • Robert Javier
  • Khanh Tran
  • Haley Nanas
  • Thomas Decker
  • Haroutun Haroutunian
  • Flavio Mendez
  • Nathan Speer
Alignment, Engagement and Contributions to the priorities of NASA’s Mission Directorates
Both NASA and the National Research Council (NRC) have concluded that existing test and evaluation methods are insufficient for advanced AI systems, and have called for R&D of innovative tools and methods to provide guaranteed behavior in unpredictable civil aviation contexts. While future autonomous systems are needed with higher degrees of autonomy, to fly safely in the National Air Space (NAS) and allow for Urban Air Mobility (UAM) requires certifiable software and, importantly, methods that allow for testing and validating software. This project responds to this demand by providing design of modular and customizable pallets, so that complex and new sensor perception and cognition algorithms can be tested and validated instantly on low-cost common form-factor UAVs. The resulting smart pallet concept advances at least the following state-of-the-art areas as requested by NASA: (1) certification process for complex non-deterministic algorithms and (2) sensors and sensor fusion algorithms.
Research Questions and Research Objectives
The research questions involved in this project include:

  • How can we ensure the reliability of sensed data and certify new sensors?
  • How can we verify the predictivity of sensory perception and cognition algorithms based on Artificial Intelligence and Machine Learning which are based on probability?
  • Simulation environments and evaluation tools are insufficient and limited to validate advanced complex probability-based algorithms on sensor fusions; how could these algorithms be certified effectively and economically?

This smart pallet project aims to tackle these questions with the following research objectives:

  • Design a portable and reconfigurable pallet for easy customization to common commercial drones;
  • Design a plug-and-playable software system for sensor data acquisition on pallet, which will work for most sensors available on the market as well as for future emerging sensors;
  • With the designed pallet system, produce publicly accessible real fight sensor data for validating algorithms and developing a more realistic simulation environment;
Research Methods
Currently we are working on the first stage of developing data acquisition software for different sensors (cameras, Lidar, IMUs, etc.). Next stage is to design the mechanical part of the pallet and to integrate together the mechanical and electrical components. After that, we plan to implement sensor fusion algorithms based on artificial intelligence on our smart pallet to enhance NASA UAS perception and cognition. The goal is to demonstrate how intelligence and sensor fusion can be embedded as a small form factor algorithm on a Raspberry PI rather than a PC. This technology will instantly enable NASA researchers to test and deploy their own algorithms and sensors on any type of commercial drones, without purchasing and customizing expensive UAVs.
Research Deliverables (Publications, Presentations and other Products)
Ultimately, the project will result in the following deliverables:

  • A proof of concept demonstration of pallet system, including both hardware and software;
  • Publicly accessible real fight sensor data for validating algorithms and developing a more realistic simulation environment; and
  • Conference publications and presentations to IEEE automatic control conference or IEEE control and decision conference.

In Fall 2020, a written report and an oral presentation have been generated about the project activities and progress.

Research Timeline
January – May 2020 —

  • Identify, acquire, and study a large set of sensors (optical, thermal, radar, Lidar, sonar, GPS, INS, etc.)
  • Identify and acquire processing units with proper interfacing ports – Raspberry Pis
  • Program Raspberry Pis to read and store raw sensor data from the sensors

May – August 2020 —

  • Program Raspberry Pis to read and store raw sensor data from the cameras
  • Examine the power management unit for the pallet
  • Recruit project team members as the previous team members graduate
  • Identify the project tasks which need to be conducted by the end of year 2020

August – December 2020 —

  • Program Raspberry Pis to read and store raw sensor data from sensors
  • Characterize the sensors using the testing data with quantitative probabilistic models on the accuracy
  • Examine quantitative measures of sensor performance and sensor data fidelity
  • Study how the location, weight, spacing, and orientation of sensors mounted on the pallet will affect vehicle platforms and environmental sensing
  • Examine how the pallet should be attached and detached easily to an air vehicle platform
  • Design electronic layout of the pallet, including interfacing connectors, adaptors, microprocessors, batteries, and wires.

January – May 2021 —

  • Assemble the programs on sensor data acquisition into one program
  • Work on an event scheduler to fully integrate the programs into one software
  • Study the sensor calibration issues
  • Manufacture the pallet and study how the location, weight, spacing, and orientation of sensors mounted on the pallet will affect vehicle platforms and environmental sensing
  • Select a drone to purchase, and test the pallet using the drone
  • Identify the mechanism by which the pallet can be attached and detached easily to the drone we select
  • Work on electronic layout of the pallet, including interfacing connectors, adaptors, microprocessors, batteries, and wires.
Are there other activities (e.g., proposals or additional projects) that you have developed or anticipate based on your NASA ARCS project?
Fall 2020:
A SBIR proposal to the NASA STTR/SBIR program has been developed and submitted in April 2020. The proposed activities build on the NASA ARCS project by designing a prototype small-scale pallet for small drones with low-cost sensors, and by providing a practical Use Case to demonstrate how the pallet concept can be used to test and validate advanced data fusion and machine learning based algorithms using actual data.

The SBIR proposal is funded for the period of September 2020 – February 2021. This grant allows our CSUN student team to join the AGNC researcher group, to ensure that the project gets done effectively and timely. Our student team will also learn from the AGNC researchers on technical knowledge as well as project management.

Spring 2021:
The SBIR project was canceled due to budgeting issue. This year we plan to collaborate with Xtensor Systems Inc. to submit a new NASA SBIR proposal. The project will be modified so that the brain of the pallet uses a Microchip processing board instead of Raspberry Pis, with the goal of gaining more flexibility and reconfigurability.

In addition, the Co-I plans to collaborate with Dr. Kourosh Sedghisigarchi at CSUN to work on a project in response to the NASA Watts on the Moon Challenge. Also, the Co-I anticipates to collaborate with Bruce Cogan from NASA Armstrong and Dr. Jack Elson from BlackSwift Inc. to work on a Venus flight simulation system. For the latter, a proposal has been submitted to the NASA Space Technology Research Program (NSTGRO20).

Impact of Project Partnership with NASA:

Shaun Mcwherter is the primary NASA Armstrong contact who is directly involved in the project work. Mr. Mcwherter has provided great suggestion and feedback on the development of the project topic and the initiation of the project process. Without this partnership with Mr. Mcwherter, this research project will not exist today. Additionally, our students are greatly motivated by the fact that they are working on a NASA-sponsored project and they are collaborating with NASA researchers.

For spring 2021, we plan to continue teaming up with Mr. Mcwherter and other researchers in NASA. Specifically, we plan to schedule a presentation from our team to our collaborators in early February and another presentation towards the end of May. Our team will benefit greatly from the presentations and the feedback received from our collaborators from NASA and other industries. Due to Covid-19, physical interaction won’t be possible.