Unified Vertex Motion Estimation for Integrated Image Stabilization and Stitching in Tractor-Trailer Wheeled Robots

Hao Liang1, Zhipeng Dong1, Hao Li1, Yufeng Yue1 Mengyin Fu1, Yi Yang1,
1Beijing Institute of Technology
Paper arXiv Video Code

Poster

teaser

UVM-VSS utilizing real-world data.

Video

Abstract

Tractor-trailer wheeled robots need to perform comprehensive perception tasks to enhance their operations in areas such as logistics parks and long-haul transportation. The perception of these robots face three major challenges: the relative pose change between the tractor and trailer, the asynchronous vibrations between the tractor and trailer, and the significant camera parallax caused by the large size. In this paper, we propose a novel Unified Vertex Motion Video Stabilization and Stitching framework designed for unknown environments. To establish the relationship between stabilization and stitching, the proposed Unified Vertex Motion framework comprises the Stitching Motion Field, which addresses relative positional change, and the Stabilization Motion Field, which tackles asynchronous vibrations. Then, recognizing the heterogeneity of optimization functions required for stabilization and stitching, a weighted cost function approach is proposed to address the problem of camera parallax. Furthermore, this framework has been successfully implemented in real tractor-trailer wheeled robots. The proposed Unified Vertex Motion Video Stabilization and Stitching method has been thoroughly tested in various challenging scenarios, demonstrating its accuracy and practicality in real-world robot tasks.

Framework

In the initial phase, camera units installed on both the tractor and the trailer undergo separate pre-calibration and individual stitching procedures to produce two partial surround views of the vehicle. Specifically, one partial surround view is constructed from the tractor-mounted camera, while the other is generated from the trailer-mounted camera.Following this, feature matching is employed to establish intra and inter-motions, thus forming a Unified Joint Motion, which serves to harmonize the motion associated with both stabilization and stitching. Then, the unified optimization process is aimed at mitigating temporal instability within the intra-frames of each camera and aligning the inter-frames between the two adjacent partial surround views. Lastly, the panoramic video is artfully crafted through mesh warping methods.

teaser

Results

Stabilization Results

teaser

Stabilization results with others

Stitching Results

teaser

Stitching results with others

Joint Stabilization and Stitching Results

BibTeX

@article{UVM-VSS,
      title = {Unified Vertex Motion Estimation for integrated video stabilization and stitching in tractor--trailer wheeled robots},
      journal = {Robotics and Autonomous Systems},
      volume = {191},
      pages = {105004},
      year = {2025},
      issn = {0921-8890},
      doi = {https://doi.org/10.1016/j.robot.2025.105004},
      url = {https://www.sciencedirect.com/science/article/pii/S0921889025000909},
      author = {Hao Liang and Zhipeng Dong and Hao Li and Yufeng Yue and Mengyin Fu and Yi Yang}
      }