Residential Collegefalse
Status已發表Published
Seamless Virtual Reality With Integrated Synchronizer and Synthesizer for Autonomous Driving
Li, He1; Han, Ruihua2,3; Zhao, Zirui2; Xu, Wei4; Hao, Qi2; Wang, Shuai5; Xu, Chengzhong1
2024-05-01
Source PublicationIEEE Robotics and Automation Letters
ISSN2377-3766
Volume9Issue:5Pages:4218-4225
Abstract

Virtual reality (VR) is a promising data engine for autonomous driving (AD). However, data fidelity in this paradigm is often degraded by VR inconsistency, for which the existing VR approaches become ineffective, as they ignore the inter-dependency between low-level VR synchronizer designs (i.e., data collector) and high-level VR synthesizer designs (i.e., data processor). This paper presents a seamless virtual reality ( SVR) platform for AD, which mitigates such inconsistency, enabling VR agents to interact with each other in a shared symbiotic world. The crux to SVR is an integrated synchronizer and synthesizer ( IS2) design, which consists of a drift-aware lidar-inertial synchronizer for VR colocation and a motion-aware deep visual synthesis network for augmented reality image generation. We implement SVR on car-like robots in two sandbox platforms, achieving a cm-level VR colocalization accuracy and 3.2% VR image deviation, thereby avoiding missed collisions or model clippings. Experiments show that the proposed SVR reduces the intervention times, missed turns, and failure rates compared to other benchmarks. The SVR-trained neural network can handle unseen situations in real-world environments, by leveraging its knowledge learnt from the VR space.

KeywordAutonomous Driving Imitation Learning Sim-to-real Virtual Reality
DOI10.1109/LRA.2024.3375266
URLView the original
Indexed BySCIE
Language英語English
WOS Research AreaRobotics
WOS SubjectRobotics
WOS IDWOS:001192358500005
PublisherIEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC445 HOES LANE, PISCATAWAY, NJ 08855-4141
Scopus ID2-s2.0-85187973741
Fulltext Access
Citation statistics
Document TypeJournal article
CollectionTHE STATE KEY LABORATORY OF INTERNET OF THINGS FOR SMART CITY (UNIVERSITY OF MACAU)
Faculty of Science and Technology
Corresponding AuthorWang, Shuai; Xu, Chengzhong
Affiliation1.University of Macau, State Key Laboratory of Internet of Things for Smart City (SKL-IOTSC), 999078, Macao
2.Southern University of Science and Technology, Department of Computer Science and Engineering, Shenzhen, 518000, China
3.The University of Hong Kong, Department of Computer Science, Hong Kong
4.Manifold Tech Limited, Hong Kong
5.Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences, Center for Cloud Computing, Shenzhen, 518000, China
First Author AffilicationUniversity of Macau
Corresponding Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Li, He,Han, Ruihua,Zhao, Zirui,et al. Seamless Virtual Reality With Integrated Synchronizer and Synthesizer for Autonomous Driving[J]. IEEE Robotics and Automation Letters, 2024, 9(5), 4218-4225.
APA Li, He., Han, Ruihua., Zhao, Zirui., Xu, Wei., Hao, Qi., Wang, Shuai., & Xu, Chengzhong (2024). Seamless Virtual Reality With Integrated Synchronizer and Synthesizer for Autonomous Driving. IEEE Robotics and Automation Letters, 9(5), 4218-4225.
MLA Li, He,et al."Seamless Virtual Reality With Integrated Synchronizer and Synthesizer for Autonomous Driving".IEEE Robotics and Automation Letters 9.5(2024):4218-4225.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Li, He]'s Articles
[Han, Ruihua]'s Articles
[Zhao, Zirui]'s Articles
Baidu academic
Similar articles in Baidu academic
[Li, He]'s Articles
[Han, Ruihua]'s Articles
[Zhao, Zirui]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Li, He]'s Articles
[Han, Ruihua]'s Articles
[Zhao, Zirui]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.