MeshRenderer¶

class gibson2.core.render.mesh_renderer.mesh_renderer_cpu.MeshRenderer(width=512, height=512, vertical_fov=90, device_idx=0, use_fisheye=False, msaa=False)¶

MeshRenderer is a lightweight OpenGL renderer. It manages a set of visual objects, and instances of those objects. It also manage a device to create OpenGL context on, and create buffers to store rendering results.

__init__(width=512, height=512, vertical_fov=90, device_idx=0, use_fisheye=False, msaa=False)¶

Parameters

width – width of the renderer output
height – width of the renderer output
vertical_fov – vertical field of view for the renderer
device_idx – which GPU to run the renderer on
use_fisheye – use fisheye shader or not

setup_framebuffer()¶: Set up RGB, surface normal, depth and segmentation framebuffers for the renderer

load_object(obj_path, scale=array([1, 1, 1]), transform_orn=None, transform_pos=None, input_kd=None, texture_scale=1.0, load_texture=True)¶

Load a wavefront obj file into the renderer and create a VisualObject to manage it.

Parameters

obj_path – path of obj file
scale – scale, default 1
transform_orn – rotation quaternion, convention xyzw
transform_pos – translation for loading, it is a list of length 3
input_kd – if loading material fails, use this default material. input_kd should be a list of length 3
texture_scale – texture scale for the object, downsample to save memory.
load_texture – load texture or not

Returns

VAO_ids

add_instance(object_id, pybullet_uuid=None, class_id=0, pose_rot=array([[1., 0., 0., 0.], [0., 1., 0., 0.], [0., 0., 1., 0.], [0., 0., 0., 1.]]), pose_trans=array([[1., 0., 0., 0.], [0., 1., 0., 0.], [0., 0., 1., 0.], [0., 0., 0., 1.]]), dynamic=False, softbody=False)¶: Create instance for a visual object and link it to pybullet

add_instance_group(object_ids, link_ids, poses_rot, poses_trans, class_id=0, pybullet_uuid=None, dynamic=False, robot=None)¶: Create an instance group for a list of visual objects and link it to pybullet

add_robot(object_ids, link_ids, class_id, poses_rot, poses_trans, pybullet_uuid=None, dynamic=False, robot=None)¶: Create an instance group (a robot) for a list of visual objects and link it to pybullet

set_camera(camera, target, up)¶

set_fov(fov)¶

get_intrinsics()¶

readbuffer(modes=('rgb', 'normal', 'seg', '3d'))¶

Read framebuffer of rendering.

Parameters: modes – it should be a tuple consisting of a subset of (‘rgb’, ‘normal’, ‘seg’, ‘3d’).
Returns: a list of numpy arrays corresponding to modes

render(modes=('rgb', 'normal', 'seg', '3d'), hidden=())¶

A function to render all the instances in the renderer and read the output from framebuffer.

Parameters

modes – it should be a tuple consisting of a subset of (‘rgb’, ‘normal’, ‘seg’, ‘3d’).
hidden – Hidden instances to skip. When rendering from a robot’s perspective, it’s own body can be hidden

Returns

a list of float32 numpy arrays of shape (H, W, 4) corresponding to modes, where last channel is alpha

clean()¶: Clean all the framebuffers, objects and instances

release()¶: Clean everything, and release the openGL context.

MeshRendererG2G¶

class gibson2.core.render.mesh_renderer.mesh_renderer_tensor.MeshRendererG2G(width=512, height=512, vertical_fov=90, device_idx=0, use_fisheye=False, msaa=False)¶

Similar to MeshRenderer, but allows rendering to pytorch tensor, note that pytorch installation is required.

__init__(width=512, height=512, vertical_fov=90, device_idx=0, use_fisheye=False, msaa=False)¶

Parameters

width – width of the renderer output
height – width of the renderer output
vertical_fov – vertical field of view for the renderer
device_idx – which GPU to run the renderer on
use_fisheye – use fisheye shader or not

render(modes=('rgb', 'normal', 'seg', '3d'), hidden=())¶

A function to render all the instances in the renderer and read the output from framebuffer into pytorch tensor.

Parameters

modes – it should be a tuple consisting of a subset of (‘rgb’, ‘normal’, ‘seg’, ‘3d’).
hidden – Hidden instances to skip. When rendering from a robot’s perspective, it’s own body can be hidden

VisualObject¶

class gibson2.core.render.mesh_renderer.mesh_renderer_cpu.VisualObject(filename, VAO_ids, id, renderer)¶

A visual object manages a set of VAOs and textures, one wavefront obj file loads into openGL, and managed by a VisualObject

__init__(filename, VAO_ids, id, renderer)¶

Parameters

filename – filename of the obj file
VAO_ids – VAO_ids in OpenGL
id – renderer maintains a list of visual objects, id is the handle of a visual object
renderer – pointer to the renderer

InstanceGroup¶

class gibson2.core.render.mesh_renderer.mesh_renderer_cpu.InstanceGroup(objects, id, link_ids, pybullet_uuid, class_id, poses_trans, poses_rot, dynamic, robot=None)¶

InstanceGroup is a set of visual objects, it is grouped together because they are kinematically connected. Robots and articulated objects are represented as instance groups.

__init__(objects, id, link_ids, pybullet_uuid, class_id, poses_trans, poses_rot, dynamic, robot=None)¶

Parameters

objects – visual objects
id – id this instance_group
link_ids – link_ids in pybullet
pybullet_uuid – body id in pybullet
class_id – class_id to render semantics
poses_trans – initial translations for each visual object
poses_rot – initial rotation matrix for each visual object
dynamic – is the instance group dynamic or not
robot – The robot associated with this InstanceGroup

render()¶: Render this instance group

Instance¶

class gibson2.core.render.mesh_renderer.mesh_renderer_cpu.Instance(object, id, class_id, pybullet_uuid, pose_trans, pose_rot, dynamic, softbody)¶

Instance is one instance of a visual object. One visual object can have multiple instances to save memory.

__init__(object, id, class_id, pybullet_uuid, pose_trans, pose_rot, dynamic, softbody)¶: Initialize self. See help(type(self)) for accurate signature.

render()¶: Render this instance