Last year’s i3 Annual Conference in Nyborg was a memorable event for many reasons, and one of those was the new reviewing style, which was significantly different from that of EC reviews in the past.

Why the change? Jakub Wejchert, EC project officer for i3, says:

"We have a responsibility not just for launching things, but also for seeing how the projects are doing from various perspectives — financial, technical, and so on. Since in the case of i3 we are not just talking about individual projects, but about an interactive clustering of them, we had to get away from the one contract/one review scenario. That's why we introduced large group reviews.

The new-style reviews are a bit like big jamborees: everybody involved demonstrates what they have being doing — by poster, demonstration, talks or discussions — and independent reviewers get to participate in all of this. In addition, there are closed sessions as well, in which reviewers give feedback and discuss specific aspects of the projects they have been assigned to.

The advantages of jamboree reviews — if we could call them that — is that they encourage synergy and sharing. Everybody can get a feel for what everybody else is doing, and people can spontaneously organise themselves to work together on topics of common interest. You just have to look at the flourishing list of workshops at Spring Days as evidence of that.

At the same time jamboree reviews have the advantage that they raise a sense of motivation and competition amongst projects. In fact the peer-to-peer reviewing, which happens naturally as part of the process, can be quite tough! For the actual reviewers themselves it has meant that they have a sense of cross-comparison, as well as being able to ask specific questions on the spot.

You know, after Nyborg most reviewers said that they much preferred this style of reviewing, as they could find out what was really going on. Some projects had to change their plans because of results coming out of others — the whole process is meant to be dynamic. This way we can really make the whole thing greater than the sum of its parts… "

Niels Ole Bernsen, coordinator of i3, sees both advantages and potential risks. It was refreshing, he writes, that

" (...) the project programme was not cast in stone. It is not, in and by itself, a virtue to stick to it for three years. The attitude towards the project programme [at the review] was entirely pragmatic or opportunistic: do something else or different if this can be justified; look for concrete exploitable or patentable aspects and go for them, even if that might mean drastic revisions of the work plan."

He also points out that there is a danger that

" (...) projects might get into a perennial process of project (re-)writing. Project plans tend to be tightly woven entities: if you change part of them you may have to change most of them. That means new deliverables, new partner roles, new time schedules and deadlines, new skills called for, and so on. This can be almost lethal to getting anything done.

And, perhaps even worse: successful project writing takes innovative thinking. The fact that a consortium successfully achieved this once does not imply that it will do so again a second time. In other words, there is a risk that a project might go from "was interesting" to "is boring"."

The (few) reactions we managed to canvass from i3 project coordinators were mixed. Some people experienced the new style of reviewing as aggressive, others didn’t agree with this. The co-NEXUS project also felt that the review was

"(...) very demanding for the projects. They had to invest a lot of time and energy into things like the presentation of the project in plenum, parallel sessions, the project stand, and the review itself. (…) We don't think it [was] worth the trouble [for the projects].”

Tim Trevan of Populate commented on the timing ofthe review:

" (...) it took place at a stage in our project when it made no practical sense at all to hold a review — halfway into the two major work packages. I strongly believe that a review at the beginning or the end of these key work-packages would have been useful, but slap in the middle? Who can review half-written sofware or a half-built prototype? Maybe in a five-year development programme that makes sense, but in a 12 -month development cycle, in my view, it does not. In fact, it distracts attention."

Kristina Höök of Persona and Thomas Rist of Magic Lounge commented more thoroughly. On the next two pages we give their accounts of the review in full.

What was different about the Nyborg reviews? The reviewing style at Nyborg differed from the traditional EC format in several ways: • The reviewers attended the entire Annual Conference, spent a lot of time at project demonstrations and poster sessions, sat in at workshop presentations, and so on. Normally reviewers prepare for a review by reading reports, and only get to see demos “in real time” at the actual review. In Nyborg reviewers had ample time to digest and discuss material before the actual review. • The actual reviews were shorter than normal. • The actual reviews were entrepreneurial in spirit. Reviewers were there to maximise the impact of project, rather than just scientific excellence per se. This meant emphasising practical, demonstrable things, while at the same time preserving the long–term vision and exploratory character. Reviewers wanted to find out which parts of projects were the most promising and would actively encourage projects towards those. • The actual reviews were strongly oriented towards aspects of innovation. If some really innovative aspect was found, the reviewers did not hesitate to ask a project to re-orient its efforts to focus on the innovation, even if that might involve drastic revisions to the existing three-year workplan.