Overview
We present a novel view synthesis method for mobile robots in dynamic environments and its application to the estimation of future traversability. Our method predicts future images for given virtual robot velocity commands using only RGB images at previous and current time steps. The future images result from applying two types of image changes to the previous and current images: 1) changes caused by different camera pose, and 2) changes due to the motion of the dynamic obstacles. We learn to predict these two types of changes disjointly using two novel network architectures, SNet and DNet. We combine SNet and DNet to synthesize future images that we pass to our previously presented method GONet to estimate the traversable areas around the robot. Our quantitative and qualitative evaluation indicate that our approach for view synthesis predicts accurate future images in both static and dynamic environments. We also show that these virtual images can be used to estimate future traversability correctly. We apply our view synthesis-based traversability estimation method to two applications for assisted teleoperation.
Visualization
Predicted images in static (first three rows) and dynamic (last five rows) environments. From left to right: previous, current and future (ground truth for view synthesis, GT) images, predicted images from baselines (SNet and DNet variants), and predicted images from our proposed dynamic-scene view synthesis approach (most right). Our method can transform the static environment depending on the robot pose change and predict the pedestrians motion.
Predicted Images by Baseline Methods and Our Method(most right)
Application Examples
GONet++ is tested with GONet on two applications as the typical atcual example.
Early Obstacle Detection
GONet++ predicts the future images on the future trajectory expected from the tele-operator's command. These predicted images are fed into GONet to estimate the traversability. As the result, GONet++ and GONet can achive the early obstacle detection to support the tele-operator. In this application example, our method predicts 4 images with virtual maximum traverling velocity vmax and the appropriate angular velocity vmax/r. Here, r is the turning radius, which is calculated by vc/wc. vc and wc is the tele-operator's command. We give an audible warning alarm, if the traversable probability for 3rd and/or 4th predicted images are less than 0.5. And, If the traversable probability for current, 1st, and/or 2nd predicted images are less than 0.5, we stop the robot by interrupting the tele-operator's command.
Multi-path Traversability Estimation
GONet++ can accept the multiple virtual commnads to predict the images on the multiple future trajectory. By feeding these predicted images into GONet like early obstacle detection, the robot can understand when and where the robot will colide with the obstacle and find the traversable area. Our method can help to avoid the obstale. In our application example, we predict 4 images on each 5 future trajectories by GONet++.
