Educational assessments evaluated
The Educational Advice and Quality Assurance department of the Faculty of Humanities has evaluated 38 recent educational assessments. What can we learn from the course of these inspections?
Every six years, tension at the programme boards rises to great heights: the educational assessments are coming up! That is when colleagues from other universities come to assess the quality of study programmes at Leiden. They do this on behalf of the NVAO and under the directions of an assessment agency such as QANU. The assessment panel talks to lecturers, students, alumni and the Faculty Board and wants to know everything down to the last detail. And if it is not satisfied, programmes have work to do.
This time, every programme at our faculty has received a positive assessment from the assessment panel. But an interesting question is: what were the programmes’ experiences with the whole assessment circus? And what did the programmes think of the support from Educational Advice and Quality Assurance? The latter was investigated by Margje Kamerling, policy officer at Educational Advise and Quality Assurance.
Why did you evaluate the process of the educational assessments?
‘We also regularly ask programmes to evaluate their own programme, to learn from that and to feedback the results to students. As a faculty, as a learning organisation, it makes sense that we also look at our internal processes when it comes to something like assessments. After all, the assessments are major happenings that involve a lot of people. They pose risks, but also offer opportunities to improve oneself.’
How did you go about it?
‘I held a survey amongst people who were involved in the assessments within the programmes. The goal was to gain more insight into the following questions: how do they look back on the support of the assessment process by the faculty, on the ways in which they prepared for that within their own programme and on the course of the eventual assessment?’
What are the most important findings of your evaluation?
‘We have gained several insights. For every assessment we organised a kick-off meeting, way in advance. People found these to be too abstract, with too many participants. Now we have changed this in the playbook. In six years’ time, we will discuss, in a smaller context and concretely, what is expected of programme boards and directors.
Quite a few lecturers were wary when the assessment started, because of the recovery processes in the previous round. But now, with the exception of a few programmes, they were actually quite satisfied: they had pleasant, constructive discussions with their assessment panels. So I think that one thing we learned, is that the assessment panels really want to contribute to the quality of education and that we can convey this from the start.
Another result of my evaluation is that the support from the Educational Advice and Quality Assurance team was appreciated. For example, the trial assessments that the team held at programmes were very well received. They did take some time: every person with whom the assessment panel was going to talk, sat together in a room for half a day. There, they listened to each other’s conversations. Everyone was thus also present when the programme boards were questioned by the assessment panels. Next, they, all together, evaluated that conversation and after that it was the lecturers’ and students’ turn. Such a ‘trial assessment’ proves to be very useful, if only because lecturers hear from students what they think of the programme and vice versa. And then immediately discuss this.’
Are there lessons to be learned from the evaluation report?
‘Prior to each visit, a bulky ‘self-evaluation rapport’ has to be written. That is something that a programme director often spends weeks sweating over, with feedback from policy staff, the vice-dean, an editor and those involved within the programme, sometimes even the entire staff. According to the survey, half of the programme directors involved, felt that writing the self-evaluation rapport had given them more insight into the strengths and weaknesses of their programme.’
What is your final assessment of the past round of assessments?
‘This time, every programme at our faculty has received a positive assessment from the assessment panel. The programmes appreciate the support from Educational Advice and Quality Assurance during this process. That is a good team result and something that all of us can be proud of. Especially the programmes, of course!’