Meandering Motion Blur with Instancing

[shadeops]

Fun one!

When motion blurring instances the objects seem to be going on a bit of a trip. In the image below, the red object is an instanced sphere, the blue has the same ActiveTransforms applied but is not an instance.

Both spheres should be following the same path but the instanced sphere is being adventurous.

Example Scene

Film "rgb"
    "integer yresolution" [ 512 ]
    "integer xresolution" [ 512 ]
    "string filename" [ "motion_blur.0219.png" ]
PixelFilter "gaussian"
    "float yradius" [ 2 ]
    "float xradius" [ 2 ]
Sampler "zsobol"
    "integer pixelsamples" [ 128 ]
Integrator "volpath"
    "integer maxdepth" [ 5 ]

Accelerator "bvh"
Transform [ -1 0 0 0 0 0 -1 0 0 1 0 0 74 -6 6 1  ]
Camera "perspective"
    "float screenwindow" [ -1 1 -1 1 ]
    "float fov" [ 30 ]

WorldBegin

AttributeBegin
    Rotate -45 0 1 0
    Rotate 40 1 0 0
    LightSource "distant"
        "float scale" [ 10 ]
AttributeEnd

MakeNamedMaterial "red"
    "rgb reflectance" [ 1 0 0 ]
    "string type" [ "diffuse" ]

MakeNamedMaterial "blue"
    "rgb reflectance" [ 0 0 1 ]
    "string type" [ "diffuse" ]

ObjectBegin "test"
AttributeBegin
    NamedMaterial "red"
    Shape "sphere"
        "float radius" [ 0.5 ]
AttributeEnd
ObjectEnd

AttributeBegin
    ActiveTransform StartTime
        Translate 73 0.4 6.0
        Rotate -50 0 0 1
        Rotate -200 0 1 0
        Rotate -144 1 0 0
        Scale 0.123457 0.123457 0.123457
    ActiveTransform EndTime
        Translate 73.4 0.4 6.0
        Rotate -30 0 0 1
        Rotate -144 0 1 0
        Rotate -65 1 0 0
        Scale 0.123457 0.123457 0.123457
    ActiveTransform All 

    ObjectInstance "test"
    
    AttributeBegin
        NamedMaterial "blue"
        Shape "sphere"
            "float radius" [ 0.5 ]
    AttributeEnd
AttributeEnd

[shadeops]

My current theory is this is due to the way the inverse transforms are applied after the interpolation.
ie)

pbrt-v4/src/pbrt/util/transform.h

Lines 458 to 463 in 47201aa

	PBRT_CPU_GPU
	Point3f ApplyInverse(Point3f p, Float time) const {
	if (!actuallyAnimated)
	return startTransform.ApplyInverse(p);
	return Interpolate(time).ApplyInverse(p);
	}

As an example of how the curved path can occur -

Right side locators are the inverse of the left side.
Moving locator on the right side is calculated by interpolating the transforms of the two left locators, then inverting.

In this example, as before the right side locators are the inverse of the left side, however the moving locator on the right is calculated by first taking the inverse of each transform, then interpolating that result.

[shadeops]

I think I narrowed this down further and at least fixed the "artifacts" that I was perceiving. The above theory was in the ballpark but not exactly right.

The issue relates to the space in which the interpolation happens,

pbrt-v4/src/pbrt/scene.cpp

Lines 370 to 376 in 47201aa

	if (CTMIsAnimated()) {
	AnimatedTransform animatedRenderFromInstance(
	RenderFromObject(0) * worldFromRender, graphicsState.transformStartTime,
	RenderFromObject(1) * worldFromRender, graphicsState.transformEndTime);
	instanceUses.push_back(
	InstanceSceneEntity(name, loc, animatedRenderFromInstance));

The interpolation has the worldFromRender applied. The quick hack that I did to address this was in the AnimatedTransform::Interpolate() to remove the worldFromRender transform, decompose both startTransform and endTransform, interpolate, then reapply the worldFromRender transform. Which caused this this to become a very expensive function, but at least validated what was causing the issue.

Master (47201aa)

AnimatedTransform::Interpolate() fix

Example Scene

Film "rgb"
    "integer yresolution" [ 256 ]
    "integer xresolution" [ 256 ]
    "string filename" [ "motion_blur_simple.png" ]
PixelFilter "gaussian"
    "float yradius" [ 2 ]
    "float xradius" [ 2 ]
Sampler "zsobol"
    "integer pixelsamples" [ 128 ]
Integrator "volpath"
    "integer maxdepth" [ 1 ]

Accelerator "bvh"

Translate 0 0 25

Camera "perspective"
    "float screenwindow" [ -1 1 -1 1 ]
    "float fov" [ 30 ]

WorldBegin

AttributeBegin
    Rotate -45 0 1 0
    Rotate 40 1 0 0
    LightSource "distant"
        "float scale" [ 10 ]
AttributeEnd

MakeNamedMaterial "red"
    "rgb reflectance" [ 1 0 0 ]
    "string type" [ "diffuse" ]

MakeNamedMaterial "blue"
    "rgb reflectance" [ 0 0 1 ]
    "string type" [ "diffuse" ]

ObjectBegin "test"
AttributeBegin
	Identity
    NamedMaterial "red"
    Shape "sphere"
        "float radius" [ 0.5 ]
AttributeEnd
ObjectEnd

AttributeBegin
	ActiveTransform StartTime
	Translate 0 5 0
	Rotate -90 0 1 0
	ActiveTransform EndTime
	Translate 0 -5 0
	Rotate 90 0 1 0
	ActiveTransform All
	
    ObjectInstance "test"
   
#    AttributeBegin
#        NamedMaterial "blue"
#        Shape "sphere"
#            "float radius" [ 0.5 ]
#    AttributeEnd
AttributeEnd

I didn't submit a PR for this because I kinda made a mess of the API by stashing the worldFromRender in AnimatedTransform 😬

[shadeops]

This can also be addressed on the scene creation side as well. For example using the current master - 315b28b we get -

But if we apply the inverse of the camera's transform in the ObjectBegin description, then reapply it prior to calling ObjectInstance we get -

Film "rgb"
    "integer yresolution" [ 256 ]
    "integer xresolution" [ 256 ]
    "string filename" [ "motion_blur_simple.png" ]
PixelFilter "gaussian"
    "float yradius" [ 2 ]
    "float xradius" [ 2 ]
Sampler "zsobol"
    "integer pixelsamples" [ 512 ]
Integrator "volpath"
    "integer maxdepth" [ 1 ]

Accelerator "bvh"

# XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
# Camera's xform
Rotate -20 1 0 0
Rotate 15 0 1 0
Translate -1.5 -3 6
# XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Camera "perspective"
    "float screenwindow" [ -1 1 -1 1 ]
    "float fov" [ 45 ]

WorldBegin

AttributeBegin
    LightSource "infinite"
AttributeEnd

MakeNamedMaterial "red"
    "rgb reflectance" [ 1 0 0 ]
    "string type" [ "diffuse" ]

MakeNamedMaterial "blue"
    "rgb reflectance" [ 0 0 1 ]
    "string type" [ "diffuse" ]

ObjectBegin "test"
    AttributeBegin
       
        # XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
        # Apply inverse of the camera's xform
        Translate 1.5 3 -6
        Rotate -15 0 1 0
        Rotate 20 1 0 0
        # XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
       
        NamedMaterial "red"
        Shape "trianglemesh"
            "integer indices" [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 
                                21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 ]
            "point3 P" [ 0.5 -0.5 0.5 -0.5 -0.5 0.5 -0.5 0.5 0.5 -0.5 -0.5 -0.5 0.5 -0.5 
                         -0.5 0.5 0.5 -0.5 -0.5 0.5 -0.5 0.5 0.5 -0.5 0.5 0.5 0.5 0.5 
                         -0.5 -0.5 -0.5 -0.5 -0.5 -0.5 -0.5 0.5 0.5 -0.5 -0.5 0.5 -0.5 
                         0.5 0.5 0.5 0.5 -0.5 -0.5 0.5 -0.5 -0.5 -0.5 -0.5 0.5 -0.5 -0.5 
                         0.5 -0.5 -0.5 0.5 0.5 -0.5 -0.5 0.5 0.5 0.5 0.5 0.5 0.5 -0.5 
                         0.5 -0.5 -0.5 -0.5 -0.5 0.5 0.5 -0.5 0.5 0.5 -0.5 -0.5 0.5 0.5 
                         0.5 -0.5 0.5 0.5 -0.5 0.5 -0.5 0.5 0.5 -0.5 -0.5 0.5 -0.5 -0.5 
                         -0.5 -0.5 -0.5 0.5 0.5 0.5 0.5 0.5 0.5 -0.5 0.5 ]
    AttributeEnd
ObjectEnd

AttributeBegin
    Rotate 90 1 0 0
    Shape "disk"
        "float radius" [ 100 ]
AttributeEnd

AttributeBegin
    ActiveTransform StartTime
        Translate 0 1.5 0
    ActiveTransform EndTime
        Translate 1.5 0.5 0
        Rotate -90 0 0 1
    ActiveTransform All

    # XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    # Reapply camera's xform
    Rotate -20 1 0 0
    Rotate 15 0 1 0
    Translate -1.5 -3 6
    # XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    ObjectInstance "test"
AttributeEnd

AttributeBegin

    ActiveTransform StartTime
        Translate 0 1.5 0
    ActiveTransform EndTime
        Translate 1.5 0.5 0
        Rotate -90 0 0 1
    ActiveTransform All

    NamedMaterial "blue"
    Shape "trianglemesh"
        "integer indices" [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 
                            21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 ]
        "point3 P" [ 0.5 -0.5 0.5 -0.5 -0.5 0.5 -0.5 0.5 0.5 -0.5 -0.5 -0.5 0.5 -0.5 
                     -0.5 0.5 0.5 -0.5 -0.5 0.5 -0.5 0.5 0.5 -0.5 0.5 0.5 0.5 0.5 
                     -0.5 -0.5 -0.5 -0.5 -0.5 -0.5 -0.5 0.5 0.5 -0.5 -0.5 0.5 -0.5 
                     0.5 0.5 0.5 0.5 -0.5 -0.5 0.5 -0.5 -0.5 -0.5 -0.5 0.5 -0.5 -0.5 
                     0.5 -0.5 -0.5 0.5 0.5 -0.5 -0.5 0.5 0.5 0.5 0.5 0.5 0.5 -0.5 
                     0.5 -0.5 -0.5 -0.5 -0.5 0.5 0.5 -0.5 0.5 0.5 -0.5 -0.5 0.5 0.5 
                     0.5 -0.5 0.5 0.5 -0.5 0.5 -0.5 0.5 0.5 -0.5 -0.5 0.5 -0.5 -0.5 
                     -0.5 -0.5 -0.5 0.5 0.5 0.5 0.5 0.5 0.5 -0.5 0.5 ]
AttributeEnd