A Five-Star Rating Scheme to Assess Application Seamlessness

Tracking #: 1164-2376

Nick Del Rio

Responsible editor: 
Guest editors linked data visualization

Submission type: 
Full Paper
Visual analytics is a costly endeavor in which analysts must coordinate the execution of incompatible visualization tools to derive coherent presentations from complex information. Distributed environments such as the Web pose additional costs since analysts must also establish logical connections among shared results, decode unfamiliar data formats, and engage with broader sets of tools that support the heterogeneity of different information sources. These ancillary activities are often limiting factors to our vision of seamless analytics, which we define as the low-cost generation and reuse of analytical resources. In this paper, we offer a theory of analytics that formally explains how analysts can employ Linked Data to maintain and leverage explicit connections across shared results as well as manage different representations of information required by visualization tools. Our theory builds on the well-known benefits of interconnected data and provides new metrics that quantify the utility of interconnected user- and task-centric, analytical applications. To describe our theory, we first introduce an extension of the W3C PROV Ontology to model analytic applications regardless of the type of data, tool, or objective involved. Next, we exercise the ontology to model a series of applications performed in a hypothetical but realistic and fully-implemented scenario. We then introduce a measure of seamlessness for any chain of applications described in our Application Ontology. Finally, we extend the ontology to distinguish five types of applications based on the structure of data involved and the behavior of the tools used. Together, our seamlessness measure and application ontology compose our Five-Star Theory of Seamless Analytics that embodies tenets of Linked Data in a form that emits falsifiable predictions and which can be revised to better reflect and thus reduce the costs embedded within analytical environments.
Full PDF Version: 


Solicited Reviews:
Click to Expand/Collapse
Review #1
By Roberto García submitted on 14/Sep/2015
Review Comment:

My concerns about connecting the proposal to end-users and user experience have been addressed as future work plans.

Review #2
By Aba-Sah Dadzie submitted on 30/Sep/2015
Review Comment:

The authors have done a good job of responding to my comments. The paper reads quite well, and the running example now does a good job of illustrating the challenges in the exercises carried out by the two "analysts", on their own and also carrying on from each other's.

One thing that needs clarifying - at the start of the conclusions: "We forged a Theory of Seamless Analytics that predicts …" - again the question I raised before, and which is actually addressed quite well in the response - is this meant to be "Seamlessness"? "Seamless" does make sense if you think a bit upside-down to what has been presented, but it contradicts the title and therefore unnecessarily causes confusion - easily interpreted as the theory is in the end turned on its head.

A few minor comments:

p.19 - "Although Chi’s effort was centered on data transformation, much like our theory, his model lacked a cost structure that could be used to establish metrics for rating or ranking visualizations." - much LIKE or UNLIKE, rather? One of the things this paper does is to try to cost the visual analysis!

p.20 - "When our theory predicts a higher cost than what is observed, we can characterize work environments where the overhead of generating and maintaining LD is not outweighed by the cost savings LD provides." - this is contradictory - surely, if the theoretical cost is higher than actual then the cost savings of LD MUST outweigh the cost of generating it?

A small number of errors and typos - an auto-check should catch these.