Why is the depth cull filter needed?

The render cache stores its data as purely as points in world space, it does not have any explicit notion of surfaces.  Thus it has to use the available point data when determining visibility and occlusion.  During the projection of the points, the only way to tell if a point should be occluded in this frame, is if another closer point maps to the same pixel (depth with a pixel is resolved using a z-buffer).  However due to gaps in the point data, sometimes a point which should be occluded will remain visible.  The causes background objects to "show through" foreground objects, making them appear less than solid.

The depth cull is a heuristic filter for trying to fix such occlusion errors.  It compares the depth values at pixels with the depth values of the neighboring pixels (pixels which didn't have a point project onto them are ignored for this calculation).  If its depth is significantly farther away than the average depth of its neighborhood, then its likely that this point should have been occluded and we invalidate it.  This filter successfully removes many points that should have been occluded, but also falsely removes some points should have been visible (especially near depth discontinuity edges).  Such false removals are acceptable because the interpolation filters can fill in the gaps thus created generally without causing any objectionable artifacts.

Instructions

  1. Start the render cache program from the button on the main page.  It will start by showing a simple environment with 7 spheres.
  2. Disable (uncheck) the "Depth Cull" checkbox in the Options panel to turn off the depth cull filter.
  3. Position the mouse over the image and move the camera left and right by pressing and holding the J and L keys.  Notice that the background of the background and farther spheres sometime are visible through the foreground spheres.  (If you get lost you can go back to the original viewpoint with the "Reset Camera" action in the Scene menu.)
  4. Now enable (check) the "Depth Cull" option and try moving the camera left and right again.  You should notice that much less of the background shows through and the objects appear more "solid".
  5. The point cache will continuous adapt for any new viewpoint, but you can freeze its state using the "Freeze Points" checkbox in the Options panel.  As you move the viewpoint now, the point density will become less and less adapted to the new viewpoints causing more artifacts.  This makes it easier to demonstrate the occlusion error artifacts that the depth cull filter is trying to correct.
  6. You can also view the raw points without (before) the interpolation filters by changing the Display pull-down box to "Raw Points".  Once you have frozen the point cloud and chosen a viewpoint where the depth cull is changing the image, this mode allows you to see exactly which points are removed (culled).  You can also see that the depth cull falsely removes a line of pixels just outside depth discontinuities (around the outside of the spheres when shown against the background).

Sample Images

without depth cull filter
Example where background points (which should be occluded) are showing through foreground objects due to gaps in the available points on the foreground object.

with depth cull filter
Example using the same data as above except that the depth cull filter has been used to remove many of the points which should have been occluded.  This improves the reconstruction and make the objects appear more "solid".