Measuring impact

measuringThe measurement of impact will vary greatly between disciplines and projects. How to evaluate the cultural impact of historical research, the societal impact of political research, or the economic impact of medical research, for example, is one of the biggest challenges associated with the impact agenda. There are no set formulas for impact evaluation, each research project needs to be considered individually. The following broad framework is designed to help researchers start this process.

Impact evaluation framework

  1. Understand the environment in which impact may occur
  2. Develop evaluation questions based on this knowledge
  3. Identify appropriate data collection methods
  4. Present the findings

1. Understand the environment in which impact may occur. It may be useful to consider the following questions/points:

  • Who are the key non-academic stakeholders (users/beneficiaries of research) in your proposed area?
  • What are the stakeholders particularly interested in?
  • What is the current situation for them? (the baseline environment from which change might be defined)
  • What is the issue or need?
  • What are the key drivers and barriers for them?
  • What might you change?
  • How and when are you going to do it?
  • Define the expected change outcomes

It may be appropriate to conduct interviews, focus groups or surveys with stakeholders in order to fully understand the impact environment.

2. Develop evaluation questions based on this knowledge. Evaluation questions should be based on the expected change outcomes defined in step 1 above.

3. Identify appropriate data collection methods. Both qualitative and quantitative techniques can have limitations – a combination of the two may result in the most convincing evidence of impact.

Qualitative techniques – Can be subjective in nature.

  • Interviews: one to one, structured or semi-structured.
  • Focus groups: Bring together a group of people to gather feedback.
  • Observations

Quantitative techniques - Can mislead if presented without a narrative or without proper consideration of factors external to the study.

  • Surveys: Online surveys can produce statistics very quickly from large groups. Test surveys on small groups first. Can incorporate some qualitative data with use of “free text” options.
  • Indicators: Selecting and monitoring a set of figures as indicators of impact. A key issue will be causation - proving a link between statistical evidence and your research.

4.  Present the findings.

  • If the impact evaluation is for a REF impact case study, consider the REF impact case study guidance and structure.
  • If the evaluation is for a funder or stakeholder, consider the level of evaluation detail needed to make an effective case.

Adapted from "How can impact be evidenced: Practical Methods?", Tony Bromley, Chapter 8 in Achieving Impact in Research, Pam Denicolo, Sage 2014.

Find out more about:

Share this page:

Website Feedback

We endeavour to provide you with all the information you should require. If there was something you were unable to find or have a suggestion for improving our site we welcome your feedback. Please fill in this simple form

Who to contact

Research Support Services Contact details

Tel - (0116) 252 2437 or (0116) 229 7763

Fax - (0116) 252 2028

E mail - RED@le.ac.uk