Reasoning about object appearance in the context of a scene

Abstract

Automated recognition of objects is complicated by occlusion. Algorithms that operate reliably when objects are in plain view typically fail when more than 50 percent of one object is obscured by another. The reason, in short, is that these algorithms fail to find the majority of the object and therefore conclude it is not present. Moreover, since previous techniques look for single objects in isolation, they cannot reason about the simple fact that one object is in front of another. Instead, the occluded portions of objects must be labeled as not found. To explain the absence of occluded features, a recognition system must use knowledge about an object's relationship to other objects. This dissertation presents a method for recognizing occluded objects in the context of a scene. Partial scene models are constructed which explain absent features in terms of occlusion by other objects. The scene modeling exploits standard computer graphics techniques: a parameterized partial 3D scene model is used to generate a prediction of how an image will appear. Recognition is then cast as a search for the scene parameters which best match predicted to observed imagery.