Skip to content

Latest commit

 

History

History
257 lines (199 loc) · 12.5 KB

blender_procedural_datasets.md

File metadata and controls

257 lines (199 loc) · 12.5 KB

\page blender_procedural_datasets Generation of Procedural Datasets with Blender

Procedurally-generated datasets of 3D models allow us to create diverse simulation environments with nearly unlimited variety. Such environments enable developers to evaluate their algorithms under various conditions that their robots might experience. Furthermore, it is a simple way of producing a large quantity of synthetic training data for applications within machine learning.

Overview

Blender is a free and open-source 3D software suite with several tools and features for creating models that can be utilized inside robotics simulators like Gazebo. In addition to manually modelling, sculpting and texturing models, Blender also enables procedural generation of datasets that can facilitate diversity in simulation environments. Furthermore, its Python API allows us to generate such datasets programmatically.

Procedural Geometry

Geometry Nodes is a system of Blender for creating and modifying geometry through node-based operations. It enables the synthesis of many model variations by combining procedural operations with randomization. Although it enables a relatively simple and fast method for creating datasets, it can be tricky to get started. Fortunately, there are several helpful resources that can help you with the creation of your own custom 3D models, e.g. an overview video, demo files, training from Blender Studio, tutorial by Blender Guru and many others.

Python Script for Generating Procedural Datasets of SDF models

Python script procedural_dataset_generator.py contains a Blender exporter of SDF models and a generator of procedural SDF dataset. The exporter outputs models that are compatible with the SDF format and the directory structure of Fuel, which can be used with any existing .blend project. The dataset generator leverages Blender's Geometry Nodes modifiers that are configured to generate countless model variations via the Random Seed input attribute (seed for the pseudorandom generation).

This script processes all selected objects in the opened Blender scene (or all existing objects if none is selected). The exporter is configurable via CLI arguments (or by adjusting default model-specific parameters). Currently, the following features are supported:

  • Export of mesh geometry as COLLADA (.dae), Wavefront (.obj) or STL (.stl) file formats.
  • Option to exclude specific Blender objects from being exported to visual or collision geometry.
  • Export separate visual and collision geometry at different levels of resolution to improve collision-checking performance with low-poly geometry.
  • Estimation of inertial properties while assuming a uniform density.
  • Optional randomization of model properties via uniform distribution, e.g. mass and surface friction.
  • Linking of PBR textures sets via symbolic links or copies.
  • Generation of thumbnails for the exported models.

Instructions

Dependencies

Blender is required to open .blend projects and run the script that employs the Python API of Blender (bpy). Versions [>=3.0, <3.3] were tested with the script, but newer versions are also expected to work.

Run the Python Script

Option A — Run Script Inside Blender

The first option is to run the Python script directly in Blender's Text Editor tab for scripting while using the default parameters that are configurable via constants at the beginning of the script. For new projects, you can follow these steps:

  1. Open the desired Blender .blend project. It is recommended to open Blender using a console to get full access to the standard output and error.
blender [blender options] file.blend
  1. Copy the procedural_dataset_generator.py Python script into a new text data block in your .blend under the Text Editor tab.
  2. Configure the default parameters of the script for your specific models via constants at the beginning of the file.
  3. Run the script using the Run script button in the panel of the Text Editor tab at the top of the screen.

@image html https://github.com/gazebosim/gz-sim/tree/gz-sim7/tutorials/files/blender_procedural_datasets/blender_instructions.png "Instructions in Blender" width=100%

Once you follow these steps and configure the script for your .blend project, you can save it and use Option B in the future.

Option B — Run an Internal Script

The second option is to run a script that is already saved inside a .blend project, i.e. saved and configured through the procedure above. Therefore, this approach is applicable only for .blend projects previously configured with the procedural_dataset_generator.py Python script. This option enables configuring the script via CLI arguments instead of modifying its contents, which is unnecessary if the project is already configured.

# Note: `procedural_dataset_generator.py` is the name of the Python script inside the `file.blend`
blender [blender options] file.blend --python-text procedural_dataset_generator.py -- [script options]

Option C — Run an External Script

It is also possible to run the Python script externally without configuring it first inside a .blend project. This option also enables configuring the script via CLI arguments, which is often required for new models.

# Note: `/path/to/procedural_dataset_generator.py` can be an absolute or relative system path
blender [blender options] file.blend --python /path/to/procedural_dataset_generator.py -- [script options]

Examples

In order to demonstrate the generation of procedural datasets, the following two .blend projects are provided:

  • rock.blend — Models of randomized rocks for Gazebo that robots can interact with
  • woodland.blend — Static environments of natural scenery with randomly scattered assets of low-poly trees, rocks, grass and flowers (these assets were adapted from Blender Studio)

You need to download these .blend projects to follow the examples. You can do that either manually on GitHub or via wget/curl.

wget https://raw.githubusercontent.com/gazebosim/gz-sim/gz-sim7/tutorials/files/blender_procedural_datasets/rock.blend
wget https://raw.githubusercontent.com/gazebosim/gz-sim/gz-sim7/tutorials/files/blender_procedural_datasets/woodland.blend

Try Demo Files

Now you can open these projects with Blender and run the scripts. Alternatively, you could run the pre-configured Python scripts for each .blend project directly from the console and configure CLI arguments as desired.

blender rock.blend --python-text procedural_dataset_generator.py -- -o sdf_models/rock
blender woodland.blend --python-text procedural_dataset_generator.py -- -o sdf_models/woodland

@image html https://github.com/gazebosim/gz-sim/tree/gz-sim7/tutorials/files/blender_procedural_datasets/demo_blender_rock.gif "Example of generating a dataset of rock SDF models" width=100%

@image html https://github.com/gazebosim/gz-sim/tree/gz-sim7/tutorials/files/blender_procedural_datasets/demo_blender_woodland.gif "Example of generating a dataset of woodland SDF models" width=100%

You can configure the script in several ways (see blender rock.blend --python-text procedural_dataset_generator.py -- -h). For example, you can set a combination of --texture-source-mode and --texture-source-value arguments to link models to (random) PBR textures.

## Configure `--texture-source-value` with a path to one of the following directories
# ├── ./                                    # <-- Here to randomly sample a texture set
#   ├── texture0/                           # <-- Here to use texture0
#     ├── *albedo*.png || *color*.png
#     ├── *normal*.png
#     ├── *roughness*.png
#     └── *specular*.png || *metalness*.png
#   ├── texture1/
#   └── ...
blender rock.blend --python-text procedural_dataset_generator.py -- --texture-source-mode path --texture-source-value /path/to/textures

Expected Output

Once the models are generated, you can see the generated directory structure for each model.

./sdf_models/
├── rock/            # Parent directory of the generated dataset for rock models
  ├── rock0/         # SDF Model directory of the 1st model
  │ ├── meshes/      # Directory for mesh geometry
  │ │ ├── collision/ # Collision geometry, defaults to `.stl`
  │ │ └── visual/    # Visual geometry, defaults to `.obj`+`.mtl`
  │ ├── thumbnails/  # Directory for the generated thumbnail
  │ ├── model.config # Meta data about the model
  │ └── model.sdf    # SDF description of the model
  ├── rock1/         # SDF Model directory of the 2nd model
  └── ...
└── woodland/          # Identical structure for the dataset of woodland models ...
  ├── woodland0/
  ├── woodland1/
  └── ...

GZ_SIM_RESOURCE_PATH Environment Variable

In order to make the models discoverable by Gazebo, you need to set the GZ_SIM_RESOURCE_PATH environment variable such that it includes the path to the parent directory of the models.

Note that the Python script will always print the adequate command with the exact path after the export of models is finished.

export GZ_SIM_RESOURCE_PATH="${PWD}/sdf_models/rock${GZ_SIM_RESOURCE_PATH:+:${GZ_SIM_RESOURCE_PATH}}"
export GZ_SIM_RESOURCE_PATH="${PWD}/sdf_models/woodland${GZ_SIM_RESOURCE_PATH:+:${GZ_SIM_RESOURCE_PATH}}"

Spawn Models in Gazebo

Hereafter, you can spawn the generated models inside Gazebo with your preferred approach, e.g. via the Resource Spawner GUI plugin. Below are some examples of Gazebo environments using the rock and woodland SDF models.

@image html https://github.com/gazebosim/gz-sim/tree/gz-sim7/tutorials/files/blender_procedural_datasets/demo_gazebo_rock.png "Example of the generated rock SDF models in Gazebo" width=100%

@image html https://github.com/gazebosim/gz-sim/tree/gz-sim7/tutorials/files/blender_procedural_datasets/demo_gazebo_woodland.png "Example of the generated woodland SDF models in Gazebo" width=100%

Every object that uses Geometry Nodes in these projects has several input attributes that can be configured. You can open the .blend projects again, navigate to the Modifier Properties tab for each object, and try tweaking them yourself.

Known Issues

Installation of Modules in Blender's Python Environment (trimesh)

The trimesh Python module is used to estimate inertial properties. This module is not included by default with the Python environment of Blender and thus needs to be installed. It is installed automatically during the first run of the Python script. However, installing a new Python module within Blender will not be possible if you use it from Snap Store. If this is the case, you must use the --python-use-system-env Blender option when running the Python script. Furthermore, you might need to run the script as sudo for the first time if the installation fails due to a lack of permission.

Future Work

The Python script is not fully featured yet. Below is a list of features that could be eventually added to improve its functionality:

  • Add support for exporting processed objects as separate <visual> and <collision> entities of a single SDF model.
  • Add support for baking textures for all processed models. Separate <visual> entities would be required to prevent overlapping of UV maps.
  • Add support for pulling PBR textures from an online source, e.g. repository.
  • Add support for exporting lights as SDF entities.
  • Add support for exporting cameras as SDF entities.