RoCHI conferences encourage submission of a broad range of papers relevant to Human Computer Interaction. Below are summarized some of the types of contributions a paper can make to HCI, and the criteria that a reviewer may use to assess each type of contribution.
Artifacts or Techniques
Artifacts or techniques concern architectures, tools, and techniques that reveal new opportunities or new ways of interacting with a range of technologies. New architectures and tools are often best evaluated in a principled, holistic fashion on the basis of what they make possible, how they do so, and what new possibilities they open. Techniques, on the other hand, are almost always evaluated formally and quantitatively. Review criteria:
- Does the paper provide context, where you clearly review what is already known and what limitations exist in knowledge about this artifact or technique?
- Does it motivate a real problem that is worth solving? For example, does it justify the beneficial value of this artifact or technique not only in isolation but within the global context of its expected uses?
- Is the artifact or technique described in sufficient detail for others to replicate it?
- Does the paper include a rigorous and convincing validation of the artifact/technique (e.g., empirical study, usability study, field study as appropriate), where it isclearly shown that the gains are of practical significance?
Theory
Theory submissions concern new models, principles, concepts, or frameworks, or important variations on those that already exist. Theoretical contributions must be validated for their novelty, importance, descriptive power, and predictive power. Review criteria:
- Is the theory novel or there is a significant variation of an existing theory? Does the theory have relevance and importance to HCI practice and/or research?
- Does the paper show how the theory is situated with respect to other theories, and how it builds upon them?
- Is the theory understandable by a general HCI practitioner as well as an expert?
- Has the value of the theory been demonstrated, e.g., by showing how it helps describe, predict and/or influence the design of an interactive application or setting?
Methodology
Methodology contributes a novel or refined method that helps HCI artifact design or evaluation. The method may be intended for use in research or development settings (or both), but the paper should be clear about the intended audience. Methodological contributions are evaluated on the basis of the novelty and utility of the new or improved method. Review criteria:
- Is the methodology novel? Does the methodology have relevance and importance to HCI practice and/or research?
- Does the paper show how the methodology builds on or contrasts with related work?
- Is the methodology described in sufficient detail so that others can use it?
- Has the value of the methodology been demonstrated,
Empirical research
Understanding users and tasks is crucial for the user-centered design. It concerns analysis of people, their behaviors, their situations and context, their perceptual, cognitive, and behavioral capabilities, and/or their interaction with their artifacts. Empirical research contributions consist of new findings based on systematically observed data. The research may be quantitative or qualitative (or mixed). The purpose of empirical contributions is, by providing new data, to reveal formerly unknown insights about human behavior and its relationship to technology. Empirical contributions are considered trustworthy when the methods that produce them are executed with rigor and precision. Identifiable confounds and bias must be avoided. Review criteria:
- Does the paper clearly describe the methodology and mechanics behind the analysis (e.g., ethnographic, field investigation, interviews, and contextual inquiry)? Is the method sound?
- Are the limitations of results mentioned?
- Do you clearly describe what new understandings were gained?
- Do the authors clearly show how the value of this new understanding is relevant to HCI, i.e., where it increases our understanding of human behavior as related to HCI, or how it adds insight into the organization of social settings, or by showing implications for design, or how it causes us to reconsider prior perspectives of human behavior and contexts within HCI, or by helping us analyze other relevant contexts?
Survey
Survey submissions concern review and synthesis of the work done in a research field with the goal of exposing trends, themes, and gaps in the literature. Survey contributions take a step back, organizing the literature of a field and reflecting on what it means. Review criteria:
- Survey contributions must not be mere catalogs of prior work. Rather, they must review and synthesize this work, extracting emergent themes or trends, and identifying gaps where new opportunities lie.
- Does the survey follow the criteria of good material organization, completeness, and thoroughness? Does the survey feature a deep synthesis, maturity of the perspective, and fairness with which other authors’ work is characterized?
- Does the survey uncover promising new areas for future work?
Applications
Interactive application submissions contribute with a novel interactive application, user interface features, or user interface design. The emphasis should be on the user interface design or evaluation. Novel application or design does not necessarily require a user study, but it does require an appropriate and reasonable validation, making clear what new possibilities they open. User interface evaluations follow the requirements of the empirical research contribution. Review criteria:
- Do the authors show what is of interest or is valuable to the CHI community? Is the design placed in context, where the expected situation, tasks and users are detailed?
- Are presented ideas validated? Validation can be by sound argumentation, reflection of experiences, user studies? For example, it can be demonstrated that an idea makes new or more functional interfaces practical or the creation of existing interface techniques dramatically easier?
- Does the paper demonstrate performance gains that clearly impact the user experience?
Argument
Argument papers contribute provocative essays. They present the author’s well-supported arguments about a topic of significant interest to a relatively broad segment of the CHI community. They have well supported claims, including consideration of other perspectives, and/or data from research or practice, if applicable. They are expected to have a stimulating effect on the CHI community. Your paper will likely include answers to the following questions:
- Does the paper clearly show that it presents a topic of highly significant interest to a relatively broad segment of the CHI audience?
- Is it provocative?
- Will it have high impact to HCI? That is, will it stimulate significant discussion and perhaps changes in our practices?
- Is the argument well founded? That is, is it well argued? Is it well supported by facts?
References
ACM-SigCHI (2014) Contributions to HCI – CHI 2014. CHI 2014 Conference. Available at: https://chi2014.acm.org/authors/contribution-types
Wobbrock, J., Kientz, J. (2016) Research Contributions in Human Computer-Interaction. Interactions 23(3), 38-44.