DOI number:  10.1016/j.vrih.2026.01.003

Affiliation of Author(s):  University of Science and Technology Beijing

Journal:  Virtual Reality & Intelligent Hardware

Funded by:  NSFC (62402281, 62361146854); FRF-TP-25-036; Tsinghua-Tencent Joint Laboratory

Key Words:  Video processing; Multimedia; Human-computer interaction

Abstract:  We present a novel 360° panoramic video conferencing system that dynamically synchronizes virtual backgrounds with real-time camera motion, addressing the limitations of static backgrounds in conventional systems. By integrating robust human segmentation, monocular visual odometry (VO), and virtual environment rendering, our method achieves seamless alignment between foreground participants and immersive 3D virtual scenes. Unlike prior approaches that suffer from foreground-background desynchronization during camera rotations or user movements, our framework estimates camera rotation in 3-DoF using a hybrid pipeline combining feature-based patch tracking and pose smoothing, while ignoring translation artifacts to maintain stability. This work bridges the gap between computational efficiency and MR-driven telepresence, offering a practical solution for next-generation virtual collaboration.

Note:  Funding: National Natural Science Foundation of China (62402281); National Natural Science Foundation of China (62361146854); Fundamental Research Funds for the Central Universities (FRF-TP-25-036); Tsinghua-Tencent Joint Laboratory for Internet Innovation Technology

Co-author:  Zian Zhou,Fanglue Zhang,Song-Hai Zhang

First Author:  Sen-Zhe Xu

Indexed by:  Journal Article

Discipline:  Engineering

Document Type:  J

Volume:  8

Issue:  2

Page Number:  219–234

ISSN No.:  2096-5796

Translation or Not:  no

Links to published journals:  https://linkinghub.elsevier.com/retrieve/pii/S2096579626000033