Consistent hidden Markov models
Date
2014
Authors
Narayana Rao Gari, Pradyumna Kumar, author
Draper, Bruce A., advisor
Beveridge, Ross, committee member
Peterson, Chris, committee member
Journal Title
Journal ISSN
Volume Title
Abstract
Activity recognition in Computer Vision involves recognizing the appearance of an object of interest along with its action, and its relation to the scene or other important objects. There exist many methods that give this information about an object. However, these methods are noisy and are independent of each other. So, the mutual information between the labels is lost. For example, an object might be predicted to be a tree, whereas its action might be predicted as walk. But, trees can't walk. However, the compositional structure of the events is reflected by the compositional structure of natural language. The object of interest is the predicate, usually a noun, the action is the verb, and its relation to the scene may be a preposition or adverb. The lost mutual information that says that trees can't walk is present in natural language. The contribution of this thesis is a method of visual information fusion based on exploiting the mutual information from Natural language databases. Although Hidden Markov Models (HMM) are the traditional way to smooth noisy stream of data by integrating information across time, they can't account for the lost mutual information. This thesis proposes an extension to HMM (Consistent HMM) that can integrate visual information to the lost mutual information by exploiting the knowledge from language databases. Consistent HMM performs better than other state of the art HMMs on synthetic data generated to simulate the real world behavior. Although the performance gain of integrating the knowledge from language databases both during training phase and run-time is better, when considered individually, the performance gain is more when the knowledge is integrated during run-time than training.
Description
Rights Access
Subject
computer vision
hidden Markov models
interacting hidden Markov models
natural language
ontology
video activity recognition