Bridging Evaluations: Inspiration from Dialogue System Research

This paper discusses methodologies for quantitatively evaluating exploratory information access environments that exploit interactive information visualization. A novel evaluation framework for such a highly interactive environment is proposed, inspired by two concepts adopted in research on spoken language dialogue systems: the PARADISE framework and the use of a simulated user. It aims to bridge two types of evaluation and to use the results of the more cost-effective one to predict the other through a version of simulation. A course of study is also discussed to make this framework feasible, which includes empirical user studies in sophisticated settings.