Ben Yang

Bimanual XR Specification of Relative and Absolute Assembly Hierarchies for Teleoperation

Ben Yang, Xichen He, Charlie Zou, Jen-Shuo Liu, Barbara Tversky, Steven Feiner

IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2026. Poster.

VR view of two nested hulls above a floor grid. A blue wireframe hull around a red shaft with two cyan gears sits inside a translucent orange hull, which also holds a pink gear on a green shaft.
Fig. 1. An absolute group, drawn as a blue wireframe hull, nested inside a relative group, drawn as an orange hull. The robot chooses where to build a relative group, while an absolute group keeps the pose the author gives it within its parent.

Abstract

We present a bimanual XR interaction approach for specifying remote assembly tasks as hierarchies of relative and absolute object constraints that specify high-level teleoperation goals for robots. Grabbing one object in each hand creates a constraint group (visualized as a hull) and groups can be nested into hierarchies. Each group can be relative (with a robot-specifiable 6DoF pose) or absolute (with an author-specified fixed 6DoF pose) in relation to its parent. A relative group specifies a subassembly that can be constructed at a location chosen by the robot software for efficiency rather than mandated by the user.

Video

Still from the video: a gear and a group of parts inside nested hulls, with two controllers below. Opens on YouTube.
Video 1. Grouping parts, nesting groups and switching between relative and absolute constraints, in VR. 1 min; opens on YouTube.

Cite

B. Yang, X. He, C. Zou, J.-S. Liu, B. Tversky, and S. Feiner, “Bimanual XR Specification of Relative and Absolute Assembly Hierarchies for Teleoperation,” in Proc. IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2026, pp. 1033–1034, doi: 10.1109/VRW70859.2026.00191.

@inproceedings{yang2026bimanual,
  title={Bimanual XR Specification of Relative and Absolute Assembly Hierarchies for Teleoperation},
  author={Yang, Ben and He, Xichen and Zou, Charlie and Liu, Jen-Shuo and Tversky, Barbara and Feiner, Steven},
  booktitle={2026 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)},
  pages={1033--1034},
  year={2026},
  doi={10.1109/VRW70859.2026.00191}
}