Loading colors for meshes from OBJ files in URDFs fails

[davidgolub]

The .obj files in the URDF file in configuration_2017-07-18/globe.urdf fail to load colors properly.

However, when I use p.loadURDF(...useMaximalCoordinates=True) or p.createVisualShape("OBJ_FILE") to load the URDF, obj files they load colors file.

Code to reproduce my issue: test.py in zip file below. (replace absolute paths to files with paths to the files in your directory)
simple_test.zip

[erwincoumans]

Can you answer the 2 questions?

The color can be

A) assigned in the URDF file using the material tag:

<link name ="rb1_marker">
    <visual>
      <origin xyz='-0.125441 0.088169 -0.129482' rpy='-1.936752 0.767602 1.007135'/>
      <geometry>
        <mesh filename="meshes/globe_sphere.dae"  scale="1 1 1" />
      </geometry>
       <material name="red">
        <color rgba="1 0 0 1"/>
      </material>
    </visual>
  </link>

B) assigned in PyBullet programmatically:

p.changeVisualShape(obUniqueId, linkIndex, rgbaColor=[r,g,b,a])

Method A and method B should work.

Did you try method A and/or B?

Which of those 2 methods don't work?

[davidgolub]

Thanks for your feedback. I tried Method A, and am able to set the color successfully, but that's not my question and I'm not using the URDF file you're referring to though. The one I'm using is in configuration_2017-07-18 that has ".obj" files as meshes. So, for example, instead of having .dae files, the mesh would reference .obj files as needed <visual> <origin xyz="-0.049149 0.089968 -0.112483" rpy="3.012065 1.349311 -3.110684"/> <geometry> <mesh filename="meshes/globe_sphere.obj" scale="1 1 1" /> </geometry> </visual>

[erwincoumans]

The URDF file doesn't have any material tag?
Can you just add the material tag inside the visual tag in the URDF file? That is method B.

 <material name="red">
        <color rgba="1 0 0 1"/>
  </material>

PyBullet currently only assigned the color using method A or method B (in the URDF).

Here an example:

<?xml version="1.0"?>
<robot name="globe">
  <link name ="rb0">
    <inertial>
      <mass value="1.0"/>
      <inertia ixx="100" ixy="100" ixz="100" iyy="100" iyz="100" izz="100"/>
    </inertial>
    <visual>
      <origin xyz='0.134201 0.028644 -0.201127' rpy='1.544267 -1.265865 -1.388046'/>
      <geometry>

        <mesh filename="meshes/globe_base.obj"  scale="1 1 1" />
      </geometry>
		<material name="red">
        <color rgba="1 0 0 1"/>
  	</material>
    </visual>
    <collision>
      <origin xyz='0.134201 0.028644 -0.201127' rpy='1.544267 -1.265865 -1.388046'/>
      <geometry>
        <mesh filename="/cvgl2/u/golubd/GibsonInteractive/exploratory/articulated_objects_db/data/objects/globe/meshes/globe_base.obj"  scale="1 1 1" />
      </geometry>
    </collision>
  </link>
  <joint name="j_0_1" type="revolute">
    <parent link="rb0"/>
    <child link="rb1"/>
    <origin xyz="0.112827626961 0.0769681379313 -0.26023652027" rpy="0 0 0 "/>
    <axis xyz="0.0692656093156 -0.417084054731 0.906224677801"/>
    <limit effort="30" velocity="1.0" lower="-3.1416" upper="3.1416"/>
  </joint>
  <joint name="j_0_fixed" type="fixed">
    <parent link="rb1"/>
    <child link="rb1_marker"/>
      <origin xyz="-0.184778028883 -0.000950895735824 0.0135191242097" rpy="0.96920987265 0.338639521178 -2.21123982949"/>
    <limit effort="30" velocity="1.0" lower="-3.1416" upper="3.1416"/>
  </joint>
  <link name ="rb1">
  </link>
  <link name ="rb1_marker">
    <visual>
      <origin xyz='-0.049149 0.089968 -0.112483' rpy='3.012065 1.349311 -3.110684'/>
      <geometry>
        <mesh filename="meshes/globe_sphere.obj"  scale="1 1 1" />
      </geometry>
			<material name="red">
        <color rgba="1 0 0 1"/>
  		</material>
    </visual>
    <collision>
      <origin xyz='-0.049149 0.089968 -0.112483' rpy='3.012065 1.349311 -3.110684'/>
      <geometry>
        <mesh filename="meshes/globe_sphere.obj"  scale="1 1 1" />
      </geometry>

    </collision>
  </link>
</robot>

[davidgolub]

Ah I see, that explanation helps. No, the URDF file does not have materials, and instead relies on meshes which reference ".obj" files, which in turn reference the ".mtl" files. So if the ".obj" file references two ".mtl" files (i.e., for the globe_base.obj file, using the command mtllib globe_base.mtl), it won't load the content in the mtl files?

[davidgolub]

And if that's the case, is it possible to replace the visual shape of a link with an output of p.createVisualShape(...)?

Currently p.changeVisualShape(obUniqueId, linkIndex, ...) seems to only support outputs from p.loadTexture (which I don't think support .obj files), but since p.createVisualShape loads all of the colors fine, would be great to just replace the visual shape of the link above with its output.

[erwincoumans]

What is the issue of adding the material field with color in the URDF file?

[davidgolub]

When I generate a .obj file from a .dae file, it automatically generates multiple .mtl files that look like this: # Blender MTL File: 'None'

Material Count: 2

newmtl Material_0.001
Ns 0.000000
Ka 1.000000 1.000000 1.000000
Kd 0.640000 0.640000 0.640000
Ks 0.000000 0.000000 0.000000
Ke 0.000000 0.000000 0.000000
Ni 1.000000
d 1.000000
illum 2
map_Kd globedone01.jpg

newmtl Material_001.001
Ns 96.078431
Ka 1.000000 1.000000 1.000000
Kd 0.015080 0.166336 0.373760
Ks 0.500000 0.500000 0.500000
Ke 0.000000 0.000000 0.000000
Ni 1.000000
d 1.000000
illum 2

Happy to add it into the URDF file if you have suggestions on how to automatically convert .mtl files into materials in .urdf files.

[erwincoumans]

If you run sdf2obj over those obj files, it will convert the mtl colors into the SDF materials automatically.
See data/threecubes for some generated example.

I also just created a pull request that adds flags for loadURDF/loadSDF that uses the colors from MTL. pybullet.URDF_USE_MATERIAL_COLORS_FROM_MTL and pybullet.URDF_USE_MATERIAL_TRANSPARANCY_FROM_MTL. Still, this will only use one color/material per link.
So if you have an OBJ with multiple parts and material colors, you still have to manually split the OBJ into separate objects each with their own color. You can use sdf2obj for that.

cd build3
./premake4_linux64 gmake
cd gmake
make App_obj2sdf
../../bin/App_obj2sdf_* --fileName="org.obj"
#now rename and load the newsdf.sdf file

[davidgolub]

Great thank you! Appreciate your help and continued discussion.

[zhaodong-wang]

Hi @erwincoumans
I encountered similar issues (might be different issue) today. I was trying to reran threecubes_org.py, however I got the following image:

Do you have any idea why I got this?

Some information:

pybullet build time: Dec 22 2019 01:51:18
Version = 4.1 ATI-3.0.68
Vendor = ATI Technologies Inc.
Renderer = AMD Radeon Pro 555 OpenGL Engine
b3Printf: Selected demo: Physics Server
startThreads creating 1 threads.
starting thread 0
started thread 0
MotionThreadFunc thread started
/opt/homebrew/lib/python3.7/site-packages/pybullet_data
numActiveThreads = 0
stopping threads
Thread with taskId 0 exiting
Thread TERMINATED
destroy semaphore
semaphore destroyed
destroy main semaphore
main semaphore destroyed

[erwincoumans]

It is expected. If you want multiple materials, you need to split the obj into parts, each with their own color/texture. That is what threecubes.py does (it loads the sdf file). Make sure to use p.GUI instead of p.DIRECT in there.

[zhaodong-wang]

@erwincoumans
I see, thanks for letting me know! Now I am able to see the three cubes in different colors using threecubes.py. However, I got the following error (although the graph still shows up)

pybullet build time: Dec 22 2019 01:51:18
Version = 4.1 ATI-3.0.68
Vendor = ATI Technologies Inc.
Renderer = AMD Radeon Pro 555 OpenGL Engine
b3Printf: Selected demo: Physics Server
startThreads creating 1 threads.
starting thread 0
started thread 0
MotionThreadFunc thread started
b3Printf: b3Warning[examples/SharedMemory/b3PluginManager.cpp,272]:

b3Printf: Warning: couldn't load plugin eglRendererPlugin

b3Printf: b3Warning[examples/SharedMemory/b3PluginManager.cpp,275]:

b3Printf: Error: dlopen(eglRendererPlugin, 10): image not found

b3Printf: b3Warning[examples/SharedMemory/PhysicsClientSharedMemory.cpp,1368]:

b3Printf: custom plugin command failed

Is this expected or we can just ignore it?

[erwincoumans]

comment-out the line about egl and ignore it. Did you compile pybullet using pip?
If not, try to use it if you want to get rid of that warning.

[zhaodong-wang]

Got it. I commented out it and now it works fine. I was using pip install pybullet to install pybullet. I will just ignore that for now. Thank you so much!

[zhaodong-wang]

The obj2sdf is great. It might be even better if there is a urdf2sdf tool that splits the obj while preserving other properties in the original urdf file.

[deepakraina99]

What is the issue of adding the material field with color in the URDF file?

@erwincoumans The issue is this (#2955). Sometimes it doesn't give a very realistic feeling. We are accustomed to seeing specific robots with specific visuals in real environment. Users would like to see the same visuals in a simulated environment. It would be good if this capability will be added to PyBullet. Thanks.