News Archive

“Crossroads” Evaluated

One of the main aims of the CEMEC project is to produce a travelling exhibition – starting in Amsterdam in September 2017, with the opening of the Crossroads exhibition. Another goal of the CEMEC project is to evaluate these exhibitions, so that the results from these evaluations may be used to make the exhibition at the next venue even better – and that is my job!

At our last partner meeting in Brussels, I presented some preliminary results of these evaluations to our partners, and I would also like to share some of the things we learned here. Please note, these results are still preliminary because the exhibition (and the research) is still ongoing! The image below shows the different types of methods that were used to study “Crossroads” from many different angles. At the bottom we see a very basic, quantitative layer which mainly gives us a lot of information, but which does not go into a lot of detail. More detailed information is provided by the top layers (observations and interviews), which give us more qualitative information, but by fewer participants. The combination of these methods allows us to provide a clear picture of what people thought of the “Crossroads” exhibition.

General satisfaction

We can start at a very basic level, which is a question about general satisfaction on the exhibition, which was (at this point) answered by more than 2000 people.

 

As the picture shows: in general – the visitors are quite happy with the exhibition. Most visitors rate the exhibition in the “Good” to “Very good” area. The average grade that people gave to the exhibition is a 7.6/10, which is quite good.

In order to understand more about why people felt that way, we handed out questionnaires to visitors, from which we gained important information regarding their satisfaction. It seems that most people really enjoy the objects in the exhibition, and especially the jewellery and the textiles. Also, people love the design of the exhibition – they think it is very beautiful. In the first room of the exhibition, a floor projection of a map of Europe showing the migrations was shown. This was a favourite among our visitors too. People wrote: “I thought the first projection on the migrations was fascinating, well done!”, calling the projection “extraordinarily interesting”, “very illustrative” and “very enlightening”.

Of course not everybody was happy about everything. Quite some people mentioned the fact that the text signs were not that easy to read, and some parts of the exhibition were very dark. Also, people did not really like the projections of the travellers on the floor – they felt these lacked information, which confused them. These are definitely things the CEMEC project is hoping to improve in the next venue!

Cross Culture Timeline

As you may remember, the CEMEC project also produced an interesting on-gallery digital tool for the purpose of the exhibition: the Cross Culture Timeline (the CCT). I have written about the testing of this project before, and I would like to share some of the results from the Amsterdam venue here with you, since they present an interesting case.

Let’s start with the basic ratings again, from more than 2000 people:

 

Now, these look quite different from the ones we saw before, from the whole exhibition, right? There seem to be about as many people who find the CCT “very good” as people who rate it as “very poor”. That’s interesting! Why would we have such ‘schizophrenic’ ratings for this application?

In order to understand these ratings, I would like to make use of a concept from museum evaluation science which, by now, is already quite old – but, I think, still quite relevant. They are the concepts of “attraction power” and “holding power”, you can learn more about them here. Attraction power, in the museum, is about how many people stop at a specific element. Visitors decide to stop somewhere, based on the amount of time and energy it will cost them, compared to what they will get out of it. Holding power is about the amount of time people spend using that element (looking at a showcase, for example) once they have stopped.

Let’s look at the CCT from that perspective. First: attraction power. We asked about 2000 people whether they made use of the CCT:

 

About 60% of them did. In comparison – about 85% of the people said they’d seen the introduction movie, so this figure is quite a bit lower. Why is that the case? I’ve found some reasons in the questionnaires. First of all: the CCT is positioned all the way at the end of the exhibition. I’m not sure if you’ve ever heard of the phenomenon of museum fatigue, but.. well, it is a thing! 😉 By this time, people are tired, have already seen a lot, and another thing (especially an interactive thing) is just too much. And, even worse, you can already see the exit from the room, and research has shown that, as soon as they see the exit, people inexorably move toward it. 😉 (You can read some more about this theory of ‘exit gradient’ in this handy book by Stephen Bitgood). Another reason why people did not engage: the application was already in use by someone else, and it was simply too busy. And finally, and most importantly, the user interface of the CCT is, at this moment, quite complex. Simple things, such as tapping and swiping, are not working well, and basic interface elements are too complicated for visitors to easily understand. This means that, when visitors do want to interact, they give up very quickly (and with lots of frustration) because it simply does not work well.

So looking at the attraction power of the CCT we can say, well, it’s quite terrible 😉 But then, and here’s the magic, we look at the holding power of the CCT – and it’s actually quite good! If the visitors engage, they spend an average of about 3.7 minutes using the CCT – and that’s quite a lot, compared to the average time spent in the exhibition (which is 30-60 minutes). So even though it is all the way at the end, with the exit in sight, and the interface is continuously working against them, visitors spend quite some time using the CCT!

And it’s my hypothesis that these schizophrenic ratings originate here: the very low ratings by the people who are very frustrated by the interface that is not quite working as it should, and then the same amount of higher ratings, from the visitors who persist, in spite of all these issues, and find the CCT an amazing tool!

But what could it be, that amazing quality in the CCT, that makes visitors stay and rate it so highly? From the 20 in-depth interviews that I undertook for this research, we may discover a clue. The first reason that is often heard (in the questionnaires too), is that people find the CCT an incredible tool to discover the different connections between the objects – they love how it visualises these relations between different times and artefacts so well. According to them, the CCT is of great added value to the exhibition, since this digital tool is much more equipped to offer this type of information, rather than a physical exhibition itself.

But secondly, and more importantly, the CCT not only offers this information, it also offers the opportunity for visitors to discover it at their own pace. Rather than a closed exhibition-narrative, the CCT functions as an open database, where visitors may discover what they find interesting in their own way. One participant said: “It’s much more interactive than walking past endless showcases – now you can decide for yourself about which objects you want to know more”. In my observations I noticed this mechanism happening – a German family of four explored the application together, and immediately moved the map towards their hometown to see if any objects were found near their house. This illustrates the concept: visitors create meaning by looking for things that they can relate to – their experiences become meaningful once they find information that is relevant to them (read more in Nina Simon’s book, The Art of Relevance).  The current CCT is definitely a small step in the right direction. Although it may need some improvements on the usability front, these early evaluations show that it definitely has potential!

Next, the CEMEC travelling exhibition will move to Athens. Visitor evaluations will also be carried  out in Athens – so hopefully we’ll be able to tell you more soon!

by Inge Kalle-den Oudsten, PhD student at the University of Amsterdam.