iGibson: Interactive Simulation of Large Scale Virtualized Realistic Scenes for Robot Learning

iGibson is a simulation environment providing fast visual rendering and physics simulation based on Bullet. iGibson is equipped with fifteen fully interactive high quality scenes, hundreds of large 3D scenes reconstructed from real homes and offices, and compatibility with datasets like CubiCasa5K and 3D-Front, providing 12000+ additional interactive scenes. Some of the features of iGibson include domain randomization, integration with motion planners and easy-to-use tools to collect human demonstrations. With these scenes and features, iGibson allows researchers to train and evaluate robotic agents that use visual signals to solve navigation and manipulation tasks such as opening doors, picking up and placing objects, or searching in cabinets.

@article{xia2020interactive,
           title={Interactive Gibson Benchmark: A Benchmark for Interactive Navigation in Cluttered Environments},
           author={Xia, Fei and Shen, William B and Li, Chengshu and Kasimbeg, Priya and Tchapmi, Micael Edmond and Toshev, Alexander and Mart{\'\i}n-Mart{\'\i}n, Roberto and Savarese, Silvio},
           journal={IEEE Robotics and Automation Letters},
           volume={5},
           number={2},
           pages={713--720},
           year={2020},
           publisher={IEEE}
         }

@INPROCEEDINGS{shenigibson,  
           author={Shen, Bokui and Xia, Fei and Li, Chengshu and Martín-Martín, Roberto and Fan, Linxi and Wang, Guanzhi and Pérez-D’Arpino, Claudia and Buch, Shyamal and Srivastava, Sanjana and Tchapmi, Lyne and Tchapmi, Micael and Vainio, Kent and Wong, Josiah and Fei-Fei, Li and Savarese, Silvio},  
           booktitle={2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},   
           title={iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes},   
           year={2021},  
           volume={},  
           number={},  
           pages={7520-7527},  
           doi={10.1109/IROS51168.2021.9636667},
          }

@InProceedings{pmlr-v164-li22b,
	  title = 	 {iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks},
	  author =       {Li, Chengshu and Xia, Fei and Mart\'in-Mart\'in, Roberto and Lingelbach, Michael and Srivastava, Sanjana and Shen, Bokui and Vainio, Kent Elliott and Gokmen, Cem and Dharan, Gokul and Jain, Tanish and Kurenkov, Andrey and Liu, Karen and Gweon, Hyowon and Wu, Jiajun and Fei-Fei, Li and Savarese, Silvio},
	  booktitle = 	 {Proceedings of the 5th Conference on Robot Learning},
	  pages = 	 {455--465},
	  year = 	 {2022},
	  editor = 	 {Faust, Aleksandra and Hsu, David and Neumann, Gerhard},
	  volume = 	 {164},
	  series = 	 {Proceedings of Machine Learning Research},
	  month = 	 {08--11 Nov},
	  publisher =    {PMLR},
	  pdf = 	 {https://proceedings.mlr.press/v164/li22b/li22b.pdf},
	  url = 	 {https://proceedings.mlr.press/v164/li22b.html},
	  abstract = 	 {Recent research in embodied AI has been boosted by the use of simulation environments to develop and train robot learning approaches. However, the use of simulation has skewed the attention to tasks that only require what robotics simulators can simulate: motion and physical contact. We present iGibson 2.0, an open-source simulation environment that supports the simulation of a more diverse set of household tasks through three key innovations. First, iGibson 2.0 supports object states, including temperature, wetness level, cleanliness level, and toggled and sliced states, necessary to cover a wider range of tasks. Second, iGibson 2.0 implements a set of predicate logic functions that map the simulator states to logic states like Cooked or Soaked. Additionally, given a logic state, iGibson 2.0 can sample valid physical states that satisfy it. This functionality can generate potentially infinite instances of tasks with minimal effort from the users. The sampling mechanism allows our scenes to be more densely populated with small objects in semantically meaningful locations. Third, iGibson 2.0 includes a virtual reality (VR) interface to immerse humans in its scenes to collect demonstrations. As a result, we can collect demonstrations from humans on these new types of tasks, and use them for imitation learning. We evaluate the new capabilities of iGibson 2.0 to enable robot learning of novel tasks, in the hope of demonstrating the potential of this new simulator to support new research in embodied AI. iGibson 2.0 and its new dataset are publicly available at http://svl.stanford.edu/igibson/.}
	}

News

2021.9 iGibson 2.0 Accepted at CoRL 2021 : [Paper] [Arxiv Version]
2021.7 iGibson 1.0 Accepted at IROS 2021 : [Paper] [Arxiv Version]
2021.7 iGibson 2.0 Released : iGibson 2.0 is a new version of our open source robot simulation environment. It implements kinematic and non-kinematic object states (temperature, wetness level, cleanliness level, etc.) and sampling functionalities to facilitate development of embodied agents for various household activities.
2021.7 BEHAVIOR Challenge Announced: BEHAVIOR is a challenge in simulation where embodied agents make continuous full-body control decisions based on sensor information. Agents need to navigate and manipulate the simulated environment with the goal of acomplishing 100 household activities Check out the challenge page for more details.
2021.2 iGibson Challenge 2021 Announced: We are annoucing iGibson Challenge 2021, hosted with Embodied AI workshop at CVPR 2021. This challenge features interactive and social navigation in indoor environments. Check out the challenge page for more details.
2020.12 iGibson and iGibson Dataset v1 Released: This release is a major update of iGibson simulation environment and a new dataset. We release a large dataset that consists of 15 fully interactive scenes and 500+ object models annotated with material and physical properties on top of PartNet and Motion Dataset. It also introduces new features such as physically based rendering, 1-beam and 16-beam LiDAR, domain randomization, motion planning, tools to collect human demos and more! For more details, please refer to our arxiv preprint.
2020.06 End of our CVPR20 sim2real challenge with iGibson! Thanks all the participants, specially team inspir.ai, the winner of this year’s edition. Their solution transferred best between iGibson simulation and the real apartment achieving maximum scores in visual navigation with and without obstacles.
2020.04 iGibson Dataset v0.05 Released: This release include the simulation environment, ten houses annotated with interactive objects of five categories, and one house fully annotated to be interactive and with selected textures. We include documentation with code examples and baselines of navigation agents with reinforcement learning state-of-the-art algorithms.
2020.02 Beginning of the simulation phase of our CVPR Challenge "Sim2Real Challenge with Gibson": Do you want to see your visual navigation algorithm run on a real robot but you don't want to deal with the real world setup? Participate in our CVPR20 challenge! We will test the best entries from a first simulation-only phase on our own Locobots in a real apartment. If you want to participate, follow the instructions in the challenge page.
2020.01 Paper Accepted at RA-L and ICRA2020: [Paper] [Arxiv Version]

Features

Blue indicates iGibson 2.0 features and red indicates iGibson 1.0 features.

Object States

iGibson 2.0 supports object states, including temperature, wetness level, cleanliness level, and toggled and sliced states, necessary to cover a wider range of tasks.

Predicate Logic Functions

iGibson 2.0 implements a set of predicate logic functions that map the simulator states to logic states like Cooked or Soaked.

Generative System

Given a logic state, iGibson 2.0 can sample valid physical states that satisfy it. It can generate infinite instances of tasks with minimal human effort.

VR Interaface

iGibson 2.0 includes a virtual reality (VR) interface to immerse humans in its scenes to collect demonstrations.

Interactive Scenes

We released iGibson dataset, which consists of 15 fully interactive scenes and 500+ object models.

Compatibility with Large Scene Datasets

Support for 12000+ scenes from CubiCasa5K and 3D-Front with full interaction. Backward compatibility GibsonV1's 572 buildings.

Robot Models

of the most common real robots and AI agents: Fetch and Freight, Husky, TurtleBot v2, Locobot, Minitaur, JackRabbot, a generic quadrocopter, Humanoid and Ant.

Physically Realistic Articulated Objects

Additional annotation of materials and dynamic properties on top of PartNet and Motion Dataset for more realistic simulation.

Fast Physics-Based Rendering and Physics

Achieving more than 100 fps with full-physics on fully-interactive scenes. More than 400 fps for rendering only.

Motion Planning and Human-iG Interface

Built in motion planners (e.g. RRT, BiRRT, LazyPRM) for arm and base. Intuitive Human-iG interface for efficient demonstration collection.

Multi-Agent Support

to train in collaborative or adversarial setups, where an arbitrary number of agents can see and interact with each other.

Reinforcement Learning Baselines

Reinforcement learning starter code for navigation tasks with visual and LiDAR signals using SAC.

Comparison to other environments:

[iGibson, 1] [Gibson, 2] [Habitat, 3] [SAPIEN, 4] [AI2-THOR, 5] [VirtualHome, 6] [ThreeDWorld, 7]

Download iGibson Dataset

iGibson 1.0 Dataset of Scenes: Dataset of 15 fully interactive scenes based on real-world homes, for robots to train in navigation and manipulation. We also include back-compatibility with the hundreds of 3D reconstructed scenes from our previous simulator, Gibson. Additionally, we provide compatibility to the scenes from CubiCasa5K and 3D-Front with additional 12000+ interactive scenes.
iGibson 2.0 Dataset of Scenes: New versions of the 15 fully interactive scenes, more densely populated with objects.
BEHAVIOR Object Dataset: Dataset of object models annotated with physical and semantic properties used by the BEHAVIOR benchmark.
Due to artists' copyright, the datasets are encrypted and allowed only to be used with iGibson. You can request a key with this form.

You can download all these scenes following the instructions here. Additional assets (robots, objects, ...) necessary to run iGibson can be obtained following the instructions here.

Documentation

The documentation for the current and previous versions of iGibson can be found here [1.0] [2.0]. The documentation includes multiple code examples and snippets to help you develop your own solutions, or modify the code for your project. If what you are looking for is not in the documentation, check the issues section of our github repository.

Details of iGibson are presented in the following papers. Consider citing them if you use iGibson in your research.

"iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks, by Chengshu Li*, Fei Xia*, Roberto Martín-Martín*, Michael Lingelbach, Sanjana Srivastava, Bokui Shen, Kent Vainio, Cem Gokmen, Gokul Dharan, Tanish Jain, Andrey Kurenkov, Karen Liu, Hyowon Gweon, Jiajun Wu, Li Fei-Fei, Silvio Savarese (* equal contribution)
[Paper] [Arxiv Version] [Bibtex]
"iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes", by Bokui Shen*, Fei Xia*, Chengshu Li*, Roberto Martín-Martín*, Linxi Fan, Guanzhi Wang, Shyamal Buch, Claudia Pérez-D'Arpino, Sanjana Srivastava, Lyne P. Tchapmi, Micael E. Tchapmi, Kent Vainio, Li Fei-Fei, Silvio Savarese. (* equal contribution)
[Paper] [Arxiv Version] [Bibtex]
"Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environments.", by Fei Xia, William B. Shen, Chengshu Li, Priya Kasimbeg, Micael Edmond Tchapmi, Alexander Toshev, Roberto Martín-Martín, and Silvio Savarese. IEEE Robotics and Automation Letters and ICRA 2020.
[Paper] [Arxiv Version] [Bibtex]

Projects Developed based on iGibson

full list

HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulators (CoRL 2019)

Chengshu (Eric) Li, Fei Xia, Roberto Martín-Martín, Silvio Savarese

[website] [paper]

Interactive Gibson: Benchmark for Interactive Navigation in Cluttered Environments (RA-L and ICRA 2020)

Fei Xia, William B. Shen, Chengshu Li, Priya Kasimbeg, Micael Tchapmi, Alexander Toshev, Roberto Martı́n-Martı́n, Silvio Savarese

[website] [paper]

ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation (ICRA 2021)

Fei Xia*, Chengshu Li*, Roberto Martı́n-Martı́n, Or Litany, Alexander Toshev, Silvio Savarese

[website] [paper]

Robot Navigation in Constrained Pedestrian Environments using Reinforcement Learning (ICRA 2021)

Claudia D’Arpino, Can Liu, Patrick Goebel, Roberto Martín-Martín, Silvio Savarese

[website] [paper]

Visuomotor Mechanical Search: Learning to Retrieve Target Objects in Clutter (IROS 2020)

Andrey Kurenkov, Joseph Taglic, Rohun Kulkarni, Marcus Dominguez-Kuhne, Animesh Garg, Roberto Martín-Martín, Silvio Savarese

[website] [paper]

Multi-Layer Semantic and Geometric Modeling with Neural Message Passing in 3D Scene Graphs for Hierarchical Mechanical Search

Andrey Kurenkov, Roberto Martín-Martín, Jeff Ichnowski, Ken Goldberg, Silvio Savarese

[website]

Interactive Simulation of Large Scale Virtualized Realistic Scenes for Robot Learning

News

Features

Object States

Predicate Logic Functions

Generative System

VR Interaface

Interactive Scenes

Compatibility with Large Scene Datasets

Robot Models

Physically Realistic Articulated Objects

Fast Physics-Based Rendering and Physics

Motion Planning and Human-iG Interface

Multi-Agent Support

Reinforcement Learning Baselines

Comparison to other environments:

Installing iGibson

Download iGibson Dataset

Documentation

Presentation

Projects Developed based on iGibson

HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulators (CoRL 2019)

Interactive Gibson: Benchmark for Interactive Navigation in Cluttered Environments (RA-L and ICRA 2020)

ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation (ICRA 2021)

Robot Navigation in Constrained Pedestrian Environments using Reinforcement Learning (ICRA 2021)

Visuomotor Mechanical Search: Learning to Retrieve Target Objects in Clutter (IROS 2020)

Multi-Layer Semantic and Geometric Modeling with Neural Message Passing in 3D Scene Graphs for Hierarchical Mechanical Search

Team

Collaborators

Alumni