Evaluators learn more about the research field, networks and application competencies

Therefore, in 2016, he was contacted by a project officer from the European Commission and invited to sign up as an evaluator. And he hasn’t regretted saying yes.

Mikkel Lodahl is an evaluator on Horizon Europe, Pillar 2 Research and Innovation Action (RIA) applications and on Creative Europe. He has read many applications and he has been involved in many evaluation processes, which he regards as thorough and very professional.

Evaluation process
As an evaluator, you are part of a database of profiles and CVs of evaluators from both academia and private companies. Each call has a project officer who establishes an evaluation team from the database.

Assessment of all applications is taken very seriously, according to Mikkel Lodahl. “When you read an application, you give it points/grades within three categories, and a written justification for each, before you meet with the others on the panel. Then you discuss it with two other evaluators, who can be either from companies or research institutions. Together, you have to agree how many points to award the application, and you have to justify your assessment in writing. Discussing an application with others is very rewarding, even though they may have a completely different opinion. You work with new evaluators on each application in a call. Finally, all evaluators for the call have a plenary meeting, after each application has also had a second reading by a fourth evaluator, who comments on the assessment, which the three original evaluators may then have to defend. So an application comes through many brains and gets many chances,” says Mikkel Lodahl.

Typical challenges
Before an application gets into the hands of an evaluator, all the applications will have been screened, and many will be rejected in the process. This means that the applications that the evaluators end up with are generally very good and rarely score less than 8 points out of 15.  So what are the typical weaknesses that lead to lower scores? In short, it’s about anchoring the project in the future and arguing various choices. Pitfalls can be that the problem is too national and lacks ‘European added value’ or there is an inadequate risk assessment of the project.

In Mikkel Lodahl's experience, a typical problem in Research and Innovation Action (RIA) applications can be failure to indicate that the project is anchored in the future. “In the field of didactic computer games, for example, there’s a challenge in that the primary school systems are very different in different European countries, so it can be difficult to show that the game can be implemented in all EU countries.”

Some applicants have not thought through potential criticism, although this is actually very necessary. As Mikkel Lodahl says, there are several assessors and they can all disagree. So you must deliver your arguments to the positive readers so that they can defend your application and you cannot be attacked. Applicants should make sure to get one or more critical readings of their application and demonstrate that they have already taken account of criticisms and are not just presenting a rosy picture.

Mikkel Lodahl has not seen many good risk mitigation measures in applications, so he recommends this as an area applicants could make a special effort to profile. And you have to make sure that you’re not dealing with just a national problem – it must be a problem that applies to most of the EU.

Benefits of being an evaluator
Mikkel Lodahl would definitely recommend applying to become an evaluator if you have a strong professional background and a good CV. He was asked himself on the basis of his LinkedIn profile and list of publications. Being an evaluator involves a lot of work. On the other hand, you learn about what is going on in the academic environment, you get an insight into the psychology of assessors and inspiration for your own applications. You also gain a large European network.

“I’ve learned a lot about what’s happening in Europe in my field, including what kind of literature others use, where the field is heading, etc. That insight is very valuable and most people spend a lot of time on it anyway,” concludes Mikkel Lodahl, who today uses his knowledge and experience as a research consultant.