MeshRenderer

MeshRenderer

class gibson2.core.render.mesh_renderer.mesh_renderer_cpu.MeshRenderer(width=512, height=512, fov=90, device_idx=0, use_fisheye=False)

MeshRenderer is a lightweight OpenGL renderer. It manages a set of visual objects, and instances of those objects. It also manage a device to create OpenGL context on, and create buffers to store rendering results.

__init__(width=512, height=512, fov=90, device_idx=0, use_fisheye=False)
Parameters

  • width – width of the renderer output

  • height – width of the renderer output

  • fov – vertical field of view for the renderer

  • device_idx – which GPU to run the renderer on

  • use_fisheye – use fisheye shader or not

setup_framebuffer()

Set up RGB, surface normal, depth and segmentation framebuffers for the renderer

load_object(obj_path, scale=array([1, 1, 1]), transform_orn=None, transform_pos=None, input_kd=None, texture_scale=1.0, load_texture=True)

Load a wavefront obj file into the renderer and create a VisualObject to manage it.

Parameters

  • obj_path – path of obj file

  • scale – scale, default 1

  • transform_orn – rotation for loading, 3x3 matrix

  • transform_pos – translation for loading, it is a list of length 3

  • input_kd – If loading texture is not successful, the color to use, it is a list of length 3

  • texture_scale – texture scale for the object, downsample to save memory.

  • load_texture – load texture or not

Returns

VAO_ids

add_instance(object_id, pybullet_uuid=None, class_id=0, pose_rot=array([[1., 0., 0., 0.], [0., 1., 0., 0.], [0., 0., 1., 0.], [0., 0., 0., 1.]]), pose_trans=array([[1., 0., 0., 0.], [0., 1., 0., 0.], [0., 0., 1., 0.], [0., 0., 0., 1.]]), dynamic=False)

Create instance for a visual object and link it to pybullet

add_instance_group(object_ids, link_ids, poses_rot, poses_trans, class_id=0, pybullet_uuid=None, dynamic=False, robot=None)

Create an instance group for a list of visual objects and link it to pybullet

add_robot(object_ids, link_ids, class_id, poses_rot, poses_trans, pybullet_uuid=None, dynamic=False, robot=None)

Create an instance group (a robot) for a list of visual objects and link it to pybullet

set_camera(camera, target, up)
set_fov(fov)
get_intrinsics()
readbuffer(modes=('rgb', 'normal', 'seg', '3d'))

Read framebuffer of rendering.

Parameters

modes – it should be a tuple consisting of a subset of (‘rgb’, ‘normal’, ‘seg’, ‘3d’).

Returns

a list of numpy arrays depending corresponding to modes

render(modes=('rgb', 'normal', 'seg', '3d'), hidden=())

A function to render all the instances in the renderer and read the output from framebuffer.

Parameters

  • modes – it should be a tuple consisting of a subset of (‘rgb’, ‘normal’, ‘seg’, ‘3d’).

  • hidden – Hidden instances to skip. When rendering from a robot’s perspective, it’s own body can be hidden

clean()

Clean all the framebuffers, objects and instances

release()

Clean everything, and release the openGL context.

MeshRendererG2G

class gibson2.core.render.mesh_renderer.mesh_renderer_tensor.MeshRendererG2G(width=512, height=512, fov=90, device_idx=0, use_fisheye=False)

Similar to MeshRenderer, but allows rendering to pytorch tensor, note that pytorch installation is required.

__init__(width=512, height=512, fov=90, device_idx=0, use_fisheye=False)
Parameters

  • width – width of the renderer output

  • height – width of the renderer output

  • fov – vertical field of view for the renderer

  • device_idx – which GPU to run the renderer on

  • use_fisheye – use fisheye shader or not

render(modes=('rgb', 'normal', 'seg', '3d'), hidden=())

A function to render all the instances in the renderer and read the output from framebuffer into pytorch tensor.

Parameters

  • modes – it should be a tuple consisting of a subset of (‘rgb’, ‘normal’, ‘seg’, ‘3d’).

  • hidden – Hidden instances to skip. When rendering from a robot’s perspective, it’s own body can be hidden

VisualObject

class gibson2.core.render.mesh_renderer.mesh_renderer_cpu.VisualObject(filename, VAO_ids, id, renderer)

A visual object manages a set of VAOs and textures, one wavefront obj file loads into openGL, and managed by a VisualObject

__init__(filename, VAO_ids, id, renderer)
Parameters

  • filename – filename of the obj file

  • VAO_ids – VAO_ids in OpenGL

  • id – renderer maintains a list of visual objects, id is the handle of a visual object

  • renderer – pointer to the renderer

InstanceGroup

class gibson2.core.render.mesh_renderer.mesh_renderer_cpu.InstanceGroup(objects, id, link_ids, pybullet_uuid, class_id, poses_trans, poses_rot, dynamic, robot=None)

InstanceGroup is a set of visual objects, it is grouped together because they are kinematically connected. Robots and articulated objects are represented as instance groups.

__init__(objects, id, link_ids, pybullet_uuid, class_id, poses_trans, poses_rot, dynamic, robot=None)
Parameters

  • objects – visual objects

  • id – id this instance_group

  • link_ids – link_ids in pybullet

  • pybullet_uuid – body id in pybullet

  • class_id – class_id to render semantics

  • poses_trans – initial translations for each visual object

  • poses_rot – initial rotation matrix for each visual object

  • dynamic – is the instance group dynamic or not

  • robot – The robot associated with this InstanceGroup

render()

Render this instance group

Instance

class gibson2.core.render.mesh_renderer.mesh_renderer_cpu.Instance(object, id, class_id, pybullet_uuid, pose_trans, pose_rot, dynamic)

InstanceGroup is one instance of a visual object. One visual object can have multiple instances to save memory.

__init__(object, id, class_id, pybullet_uuid, pose_trans, pose_rot, dynamic)

Initialize self. See help(type(self)) for accurate signature.

render()

Render this instance