Understanding images of graphical user interfaces (ui-gui)

Abstract

A fundamental problem in surveillance systems is the specification of “activities of interest”. While various activity recognition systems have been developed, they have used complex hand-coded representations. What is of interest to a particular surveillance system user can vary greatly, and the security forces using the system are not, in general, advanced computer users. This paper presents a novel paradigm for specifying and recognizing activities in a surveillance system by visualizing tracking results, and using computer vision techniques to interpret the images presented by their Graphical User Interfaces [Yu 2003]. Representation of activities of interest can be easily drawn by users. Not only is the drawing-based specification of activities of interest easier than previous approaches, but when a “rule” fires, it is easy to explain “why”, by showing the operator the associated drawings [Yu 2004]. This approach also permits a new type of system integration, where we integrate the “display” of sensors rather than trying to develop a complex communication protocol. A user study is reviewed showing the proposed approach is thousands of time faster than HMMs for event definition while it is simultaneously more accurate in event recognition.