Efficiently incorporating user feedback into information extraction and integration programs

This paper proposes a solution for users to directly provide feedback and for IE/II programs to automatically process such feedback, and shows how to automatically propagate F to the rest of P, and to seamlessly combine F with prior user feedback.

Abstract

Many applications increasingly employ information extraction and integration (IE/II) programs to infer structures from unstructured data. Automatic IE/II are inherently imprecise. Hence such programs often make many IE/II mistakes, and thus can significantly benefit from user feedback. Today, however, there is no good way to automatically provide and process such feedback. When finding an IE/II mistake, users often must alert the developer team (e.g., via email or Web form) about the mistake, and then wait for the team to manually examine the program internals to locate and fix the mistake, a slow, error-prone, and frustrating process.