Interactive Gibson is a fast simulator and a dataset for indoor navigation and manipulation. It was first released in June 2019. It allows for complicated interactions between the agent and the environment, such as picking up and placing objects, or opening doors and cabinets. This environment opens up new venues for jointly training base and arm policies, allowing researchers to explore the synergy between manipulation and navigation.
If you use Interactive Gibson Simulator or Interactive Gibson assets, please consider citing the following paper:
Fei Xia, Chengshu Li, Kevin Chen, William B. Shen, Roberto Martin-Martin, Noriaki Hirose, Amir R. Zamir, Li Fei-Fei, and Silvio Savarese. "Gibson env V2: Embodied Simulation Environments for Interactive Navigation." 2019. [Simulator] [Documentation] [Tech report (old)] [Dataset] [New tech report coming soon] [Bibtex]
Interactive Gibson dataset contains 572 buildings, 1400 floors, and 211k square meters of indoor spaces.
We augment 106 scenes with 1984 interactable CAD model alignments of 5 different object categories: chairs, desks, doors, sofas, and tables.
We can simulate articulated objects such as doors and cabinets.
Textures of CAD objects are randomized with physically-based rendering, resulting in a rich library of randomized environments.
Interactive Gibson simulate full physics yet still runs at 200+ fps. For rendering only, it can achieve close to 1000 fps.
Multiple agents can be added to the same environment, add they can see each other. This makes multi-agent learning possible
Apart from object annotations, we also incorporate scene graph annotations and render at every frame.
SAC, DDPG, and PPO baselines are provided with the environment. Pretrained models for common tasks including P2P navigation are available.
We show Gibson Env V2 is able to handle multiple agents. It can render the camera view from each agent and simulate collisions between agents.
We support rendering rgb images, surface normal, depth and segmentation masks.
In this example we show a simulated JR in Gibson Env V2. It can pull open the door in the environment. We show a third person view and the robot's view.
Objects are texture baked with path traced rendering in Gibson Env V2.
We can add multiple objects into Gibson Env V2 and simulate the interaction between agent and added objects.
Due to the photorealism of our simulator, it is easy to transfer learned policy from simulation to real world.
The table shows rendering speed (frame per second) of our environment under different rendering modes
at 256x256 resolution with full physical simulation enabled. Tested on
|Gibson V2||Gibson V1|
|RGBD, pre network
|RGBD, post network
|Surface Normal only||271.1||129.7|
The table shows the rendering only (no physics) speed of our environment compared with habitat-sim at 640x480 resolution. All benchmarks are run with a single
NVIDIA GeForce GTX 1080 Ti Graphics Card