Introduction

Modeling object categories is a challenging task owing to the many structural variations between instances of the same category. There have been many nonhierarchical approaches to modeling object categories, all with limited levels of success. Appearance-based models, which represent objects primarily by their photometric properties, such as global PGA, KPCA, fragments, SIFTs, and patches (Lowe 2004; Nayar et al. 1996; Ullman et al. 2001; Weber et al. 2000), tend to disregard geometric information about the position of important keypoints within an object. Thus, they are not well-suited for recognition in scenarios where pose, occlusion, or part reconfiguration are factors. Structure-based models, which include information about relative or absolute positions of features, such as the constellation model and pictorial structures (Felzenszwalb and Huttenlocher 2005; Fischler and Elschlager 1973;Weber et al. 2000), are more powerful than appearance-based approaches as they can model relationships between groups of parts and thus improve recognition accuracy, but are rarely hierarchical and, as such, cannot account for radical transformations of the part positions.

Very recently there has been a resurgence in modeling object categories using grammars (Jin and Geman 2006; Todorovic and Ahuja 2006; Zhu and Mumford 2006). Work by Fu (Fu 1981; You and Fu 1980) and Ohta (1985) in the 1970s and 1980s, and later by Dickinson and Siddiqi (Dickinson et al. 1992; Keselman and Dickinson 2001; Siddiqi et al. 199?) introduced these grammars to account for structural variance. Han and Zhu (2005) and Chen et al. (2006) used attributed graph grammars to describe rectilinear scenes and model clothes, but these models were hardcoded for one category of images.