Residential College | false |
Status | 已發表Published |
Weakly Supervised Monocular 3D Object Detection by Spatial-Temporal View Consistency | |
Han, Wencheng1; Tao, Runzhou2; Ling, Haibin3; Shen, Jianbing1 | |
2024-09-24 | |
Source Publication | IEEE Transactions on Pattern Analysis and Machine Intelligence |
ISSN | 0162-8828 |
Abstract | Monocular 3D object detection plays a crucial role In the field of self-driving cars, estimating the size and location of objects solely based on input images. However, a notable disparity exists between the training and inference of 3D object detectors. This discrepancy arises because during inference, monocular 3D detectors depend solely on images captured by cameras; while during training, these methods require 3D ground truths labeled on point cloud data, which is obtained using specialized devices like LiDAR. This discrepancy creates a break in the data loop, preventing the feedback data from production cars from being utilized to enhance the robustness of the detectors. To address this issue and establish a connection in the data loop, we present a weakly-supervised solution that trains monocular 3D object detectors solely using 2D labels, eliminating the requirement for 3D ground truths. Our approach considers two view consistency: spatial and temporal view consistency, which play a crucial role in regulating the prediction of 3D bounding boxes. Spatial view consistency is achieved by employing projection and multi-view consistency techniques to guide the optimization of the target's location and size. We leverage temporal viewpoint consistency to provide temporal multi-view image pairs, and we further introduce temporal movement consistency to tackle the challenge of dynamic scenes. With only 2D ground truths, our method achieves comparable performance to fully supervised methods. Additionally, our method can be employed as a pre-training method and achieves significant improvement when fine-tuned with a small proportion of fully supervised labels. |
Keyword | Monocular 3d Object Detection Production Cars Data Spatial-temporal Consistency Weakly Supervised Learning |
DOI | 10.1109/TPAMI.2024.3466915 |
URL | View the original |
Language | 英語English |
Scopus ID | 2-s2.0-85204945519 |
Fulltext Access | |
Citation statistics | |
Document Type | Journal article |
Collection | THE STATE KEY LABORATORY OF INTERNET OF THINGS FOR SMART CITY (UNIVERSITY OF MACAU) |
Affiliation | 1.University of Macau, State Key Laboratory of Internet of Things for Smart City, Department of Computer and Information Science, Macao 2.QCraft Inc, Beijing, China 3.Stony Brook University, Department of Computer Science, Stony Brook, 11794, United States |
First Author Affilication | University of Macau |
Recommended Citation GB/T 7714 | Han, Wencheng,Tao, Runzhou,Ling, Haibin,et al. Weakly Supervised Monocular 3D Object Detection by Spatial-Temporal View Consistency[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. |
APA | Han, Wencheng., Tao, Runzhou., Ling, Haibin., & Shen, Jianbing (2024). Weakly Supervised Monocular 3D Object Detection by Spatial-Temporal View Consistency. IEEE Transactions on Pattern Analysis and Machine Intelligence. |
MLA | Han, Wencheng,et al."Weakly Supervised Monocular 3D Object Detection by Spatial-Temporal View Consistency".IEEE Transactions on Pattern Analysis and Machine Intelligence (2024). |
Files in This Item: | There are no files associated with this item. |
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment