How research quality is to be improved

As previously described in an earlier news item, the University of Borås is part of a pilot programme in which the Swedish Higher Education Authority or UKÄ, for its Swedish name, Universitetskanslersämbetet, is, for the first time, examining quality systems for research. The GEM was also attended by Michael Eriksson of Chalmers University of Technology, who spoke about experiences from Chalmers’s internal project on research evaluation, which was carried out in 2017–2019. At the end, participants discussed in small groups and recorded summaries of their discussions that were given to the team working with this topic. 
The entire GEM was filmed and is available on HB-play (in Swedish)

The work on the quality system is not primarily done for UKÄ's review, said Pro-Vice-Chancellor Kim Bolton. 
“This is a good tool for us. Our quality development would benefit even more if we have a more systematic way of working with quality assurance than we have today.” 

UKÄ will review all universities. The focus is on a systematic approach to quality assurance. UKÄ does not look at the quality of research; that is the responsibility of the university. The review focuses on research but not doctoral education. 
Systematicity is a strong theme of what UKÄ is looking at, according to Kim Bolton.
“We must show evidence of what we say. We will report about the history, how we have developed our system over the years, and how the plan looks moving forward. And since the system is for the researchers, it must be accepted by them.  Everyone must be aware of it and know its purpose.”

Proposed framework 

Hanna Kantola, Research Education Coordinator and staff member involved in the project group with the self-evaluation, spoke about the work on the framework for research quality evaluation. The proposal is to follow up on the research internally every year and to carry out an external evaluation every six years. The focus is to lie on the annual follow-ups, and the information generated there should be shared throughout the university.

One proposal that is being discussed is that the six-year evaluation should be based on the university's six priority research areas, which can then be organised into one or more smaller evaluation units. It should be a flexible system so that the unit being evaluated can get the most out of the evaluation. The starting point is also that each evaluation unit should share an assessment group and an audit opportunity. 

What is to be included in the annual evaluations? The proposal contains three points: 

  • Self-assessment, according to a set template
  • Research plans, where each research group leader produces an annual plan
  • Key figures, such as publications per teaching staff, citations, external financing compared to total funding.

“Most research team leaders already have a research plan, and the idea is that there won't be much additional work for those who already have one. In a six-year cycle, these research plans will form part of the basis for the assessment. This can also function as a kind of compilation that shows how they are in line with the research area's strategy,” Kim Bolton said. 

Experience from Chalmers

Michael Eriksson from Chalmers University of Technology spoke about his experience as project manager for Chalmers’s internal projects on research evaluation between 2017 and 2019. They saw opportunities to come together and identify  strengths, weaknesses, and opportunities. The evaluation was conducted to further develop quality efforts, not to reward or punish individual institutions. 

“Initially, the project encountered some scepticism about spending additional time on evaluation. But the fact was that the attitude changed fairly quickly; they saw that the work itself was quality-enhancing,” said Michael Eriksson. 

In summary, he said that the researchers learned more than they ever expected. The evaluation work focused on important issues that may not have received much attention before. 

Work arrangement 

The work is led by Pro-Vice-Chancellor Kim Bolton and the Vice-Chancellor's Advisory Council is the steering group. A project group is to be formed to write the self-evaluation, which will be submitted to UKÄ on 19 December. 

Three theme-based workshops will be held where the Faculties and committees have appointed research group leaders as representatives. Through these workshops, self-evaluation will be examined and established around the university. The conclusions of the self-evaluation will be presented at the next GEM on 26 November in order for the university as a whole to be able to share in the result. 

In March and May 2020, UKÄ will make site visits. A preliminary opinion from UKÄ is expected in late autumn 2020. 

Text: Lina Färm
Translation: Eva Medin