Research Project

Real-Time Traffic Monitoring

(3D Object Detection and Vehicle Classification based on LiDAR Point Clouds for Real-time Traffic Flow Monitoring)

Research Team

Lead Researchers:

Xunfei Jiang, Computer Science
Bingbing Li, Manufacturing Systems and Engineering
Xudong Jia, Civil Engineering

Collaborators:

Student Team:

Rachel Gilyard*, M.S. in Computer Science (ARCS Fellow)
Jimwell Castillo*, B.S. in Computer Science
Param Ridham Desai*, B.S. in Computer Science
Luis Cedeno*, Never Khachoyan*, Jesus Casas*, Abdul Hasib Safi*, Jordan Aviles* (Computer Science Senior Design 2024-2025 Team)
Emi Anyakpor Jr.*, B.S. in Computer Science
Kousha Salimkhan*, B.S. in Computer Science
Kevil Patel*, B.S. in Information Systems
Gev Chalikyan*, B.S. in Computer Science
Nabiha Sharif* (Taft High School STEAM Magnet)
Jonathan Cordova, MS in Computer Science (ARCS Fellow)
Alexander Rose, M.S. in Software Engineering (ARCS Fellow)
Ruitao Wu, B.S. in Computer Science (ARCS Fellow)
Icess Nisce, B.S. in Computer Science (ARCS Fellow)
Chen Li, M.S. in Computer Science at Columbia University
Tongzi (Peter) Wu, Research Assistant at ARCS
Robin Rosculete, B.S. in Computer Science
Jose Flores, B.S. in Computer Science
Mahi Kachhadiya, B.S. in Computer Science
Andres Mercado, B.S. in Computer Science
Jimwell Castillo, Gisela Calva, Anthony Merzoian, Raymond Gharapeti Babayans, Alma Dollente, Kaylee Groves (Computer Science Senior Design 2023-2024 Team)
Brian Uribe, Javier Carranza, Alondra Gonzalez, Matthew Davis (Computer Science Senior Design 2022-2023 Team)
Ziaur Chowdhury, Gustavo Velasquez Sanchez, Xin Gao, Cesar Villa (Computer Science Senior Design 2021-2022 Team)

Note: names marked with an asterisk (*) indicate current students

Funding

Funding Organization:
Funding Program:

SYNOPSIS

More accurate and comprehensive traffic information can improve urban planning, reduce congestion, and enable more timely road traffic management.
Using data from a LiDAR (Light Detection and Ranging) camera and a stereo-based depth camera, machine learning algorithms can help to detect and classify vehicles.
Data are processed in real time and visualized for traffic flow analysis.

Abstract

Vehicle detection plays an important role in analyzing traffic flow data for efficient planning in intelligent transportation. Machine Learning technology has been increasingly used for vehicle detection in both 2D real-time traffic flow video and 3D point clouds. Adverse weather conditions, such as fog, rain, snow, extreme wind, and other conditions prove to be challenging for 2D vehicle detection. 3D LiDAR point clouds can be more resistant to adverse weather conditions. Most of the existing research on 3D vehicle detection is used for autonomous vehicle driving with the LiDAR cameras deployed on vehicles. There is a lack of research on real-time vehicle detection for intelligent transportation with LiDAR cameras deployed by highway/freeway. In this project, we propose to build a system that collects real-time traffic flow data through 3D LiDAR cameras, processes the 3D point cloud data for vehicle detection and classification, and provides a web-based service with vehicles detected and classified in real-time traffic flow and data visualization for statistical traffic flow data.

Motivation/Research Problem

Object detection includes the parametrization of a bounding box containing the recognized and classified object. Most research in that area has focused on two-dimensional (2D) object detection based on widely available Red-Green-Blue (RGB) or grayscale images. However, this completely leaves out the third dimension and only partly imitates the human visual system. In fact, many modern applications, like robot assistants or autonomous vehicles, are highly dependent on depth and surface data to securely navigate in three-dimensional (3D) environments through LiDAR cameras. With the rising availability of mobile 3D sensors like RGB-Depth (RGB-D) cameras and LiDAR (light detection and ranging) sensors, more depth data can be captured and processed. This allowed great progress in the field of 3D object detection (3DOD) and brought forth a variety of heterogeneous methods and solutions used for this task.

Research Questions & Research Objectives

The goal of this project is to detect and classify vehicle types and monitor the real-time traffic flow of highways in Southern California.

Use Velodyne cameras to collect 3D LiDAR point cloud data for traffic flow on highways
Automate the data calibration for matrices setting of 3D LiDAR camera
Build 3D machine learning models to detect vehicles and classify vehicle types in 3D LiDAR point cloud data
Develop a system that monitors and visualizes the real-time traffic flow of highways in South California
Apply 3D vehicle detection and classification models on real-time LiDAR point cloud traffic flow and output vehicle detection and classification results, including vehicle types and coordinates, etc.
Process the real-time vehicle detection results and generate statistical data for traffic flow, including number of vehicles in each type, traffic flow speed, density, etc.
Visualize the real-time statistical vehicle detection and classification results and predict the traffic flow on highways

Summary:

Present traffic statistical data through data visualization charts updated on the Real-Time Traffic Monitoring Website.
Increase speed and precision of 3D vehicle classification models, trained on stationary, roadside data.
Compile a dataset for further research, model improvement, and for other researchers working in traffic and transportation fields.

Research Methods

Data Acquisition for Live Traffic Stream
- Live video and point clouds are streamed from a camera and LiDAR sensor to the CSUN server to be processed by the 2D and 3D machine learning models.
- Statistical data generated from vehicle detection and classification models are visualized on the Real-Time Traffic Monitoring (RTTM) system, and traffic flow steams with bounding box inference overlays are displayed on the RTTM system.
- Historical traffic flow data are processed and stored in a Postgre database and can be accessed through the RTTM system.
Background Filtering for the PointPillars 3D Machine Learning Model
- Isolates dynamic targets by filtering out static features, such as buildings, from LiDAR data.
- Reduces the computational burden for 3D detection models.
- Mitigates the latency challenges posed by resource-intensive models.
- Increases the accuracy of the model’s inferences.
Complex-YOLO for 3D Vehicle Detection and Classification
- Performs object detection and classification by converting 3D point cloud data into a 2D Birds-Eye-View (BEV) projection and applying the YOLOv4 model for vehicle detection and classification.
- Used transfer learning approach along with normalization of the rotation angle, which enhanced the Complex-YOLO model’s performance for vehicle detection and classification.
- With normalization of the rotation angle, the best model achieved a precision of 92.20% with a recall of 84.90%.
- Extended Complex-YOLO’s classification capabilities to include additional vehicle types, and the neural network layers (number of nodes) and filter values are adjusted to accommodate the increase from three categories to nine categories.
2D Machine Learning Model
- The YOLOv8 model was chosen for its real-time, end to end inference.
- A 2D machine learning model was trained on our 2D roadside image dataset with over 2000 frames, and the model achieved a precision score of 0.944.

Research Deliverables and Products

A real-time system that presents both 2D and 3D traffic flow, with vehicles detected and labeled in the traffic streams.
3D vehicle detection and classification models.
Conference and Journal papers in the areas of Intelligent Transportation and Machine Learning for Vehicle Detection.
Publications:
- Jonathan Cordova, Xunfei Jiang, Xudong Jia, Bingbing Li, “3D Vehicle Detection in Roadside Traffic Flow using Complex-YOLO.” 23rd International Conference on Machine Learning and Applications (IEEE ICMLA 2024), Miami, FL, Dec 18-20, 2024. (Accepted for publication)
- Jimwell Castillo, Robin Rosculete, Raymond Gharapeti Babayans, Anthony Merzoian, Kaylee Groves, Gisela Calva, Alma Dollente, Xudong Jia, Bingbing Li, Xunfei Jiang. “A Real-Time Traffic Monitoring System based on YOLOv8 for Vehicle Detection and Classification,” The 15th International Conference on Green Energy and Smart Systems (IEEE GESS 2024), Long Beach, CA, USA, Nov 4-5, 2024. (Accepted for publication)
- A. Rose, I. Nisce, A. Gonzalez, M. Davis, B. Uribe, J. Carranza, J. Flores, X.-D. Jia, B.-B. Li, X.-F. Jiang, “A Cloud-based Real-Time Traffic Monitoring System with Lidar-based Vehicle Detection,” 2023 IEEE Green Energy and Smart Systems Conference (IGESSC), Long Beach, CA, USA, 2023, pp. 1-6, doi: 10.1109/IGESSC59090.2023.10321763.
- R. Wu, Z. Chowdhury, G. V. Sanchez, X. Gao, C. Villa and X. Jiang, “Real-time Vehicle Detection System for Intelligent Transportation using Machine Learning,” 2022 IEEE Green Energy and Smart System Systems (IGESSC), Long Beach, CA, USA, 2022, pp. 1-6, doi: 10.1109/IGESSC55810.2022.9955329.

Commercialization Opportunities

Application: Traffic management systems for cities, integration with autonomous vehicle edge systems, traffic pattern analysis tools for traffic operation and management
Key Values: Reduce congestion, improve urban planning decisions, increase road safety
Potential Customers: Municipalities, smart city technology providers, researchers in transportation analytics

Research Timeline

Start Date: 02/01/2021
End Date: TBD