Please enter your text to search.

Some suggestions on vocabulary

One of the things that has become clear in discussions with people leading up to the workshop is that the language we use to discuss research impact is a barrier in its own right. Confusion over terminology and interpretations of the agenda behind terminology are a significant source of problems. To help deal with this I am suggesting the following set of terms and some of the distinctions they imply as a useful way to talk about these issues.

Output: Any form of object generated directly by a research activity. This may include research papers, articles, posters, abstracts, data, samples, software, methodology, online writing, reports, and other grey literature. Outputs may be open (as defined by the Budapest declaration including the right to re-use), publicly accessible (freely viewable or obtainable), or partly accessible (if for instance they are available to subscribers, or to professional researchers, or for specific fields of use such as non-commercial use). Prior to release or publication an output is private. Outputs are the direct result of research and are therefore distinguished from outcomes.

Outcomes: Outcomes are events or changes that can be traced more or less directly to the influence of a research activity. These include; wider adoption of developed methodology, adoption of a new conceptual framework, changes to clinical guidelines, development of specific products (including medical products), changes to policy or procedures, development of a commercial enterprise, the training of research personnel, increased research efficiency. Outcomes are therefore tightly coupled to the re-use of research outputs. Outcomes can be indicator events if they are single events but may also be broader e.g. a single citation is an indicator event, an outcome would be the quantitation of all citations to a given output. Not all outcomes are quantifiable or measurable.

Indicator events: Indicator events (in the CASRAI framework) are specific occurrences that can be traced to one or more specific research activity. The generation of outputs and specific outcomes can both be indicator events. David Baker defines an indicator event as “a single recordable occurrence that changes the count of one or multiple impact indicators and that can be related to one or more research personnel or activity”.

Impact: Impact is a measure of the influence of outputs and outcomes of research. The measure may be qualitative or quantitative and may be absolute or comparative. To determine impact is to ask the question: how different would the world be if this research had not been carried out? Outcomes can provide evidence to support an assessment of impact. Outputs by contrast can not, in and of themselves, be evidence of impact. Impact may be felt in economic, social, policy, environmental, educational, or research arenas. Qualitative assessment of impact may be based on a narrative describing outcomes and indicator events. Quantitative assessment of impact may be based on one or more measures of outcomes and/or indicator events. Quantifiable outcomes and indicator events are used as proxies for impact or may be fed into a model which attempts to directly estimate or provide a comparative measure of impact.

Research Activity: Broadly any activity or endeavour that seeks to advance knowledge, understanding, or capability. Research may be directly funded or indirectly funded (often referred to incorrectly as “unfunded”). It may be undertaken by professional or amateur researchers and may or may not occur in the context of a research institution, including commercial and industrial institutions. The majority of current research impact derives from research activities carried out by professional researchers in academic and commercial research institutions.

What have I missed? What other terminology do we need to sort out?


4 Responses to “Some suggestions on vocabulary”

  1. This list covers the key topics well. I think we may have to remember to have a ‘thesaurus’ approach handy. If there is ever disagreement on a term we should first check whether the ‘concepts’ are synonymous so we know it is a labeling issue only.

  2. Another concept I’d like to submit can be thought of as a ‘parent’ concept to ‘Outputs’. The concept is ‘Contributions’. They include a sub-concept of ‘Outputs’ but also can include a sub-concept of ‘Services’. These may well be part of the overall capture of impact indicator events by stakeholders. Examples of ‘Services’ would be: Consulting/Advisory, Expert Witness, Journal Reviewing/Referring, Mentoring, Research-based Degree Supervisions, etc.). All of these ‘Service Contributions’ may not be impact indicators events but some certainly can be seen as creating outcomes that have impact. There should be a way of describing them and capturing them – if the community sees them as legitimate indicators of overall impact.

    • I guess I would probably still see these as outputs, just various different categories of output. Although I can see some value in distinguishing between output “objects” and output “services” which will have different characteristics.

      • neilchuehong says:

        I think there’s an interesting distinction between “contributions” which include things which cannot generally be directly tracked to specific “outputs”, and those which can.

        As an analogy, someone may have provided a valuable but intangible contribution to a paper that the authors want to recognise but not contributed words, corrections, direct inputs or data.

Leave a Reply