Evaluating The Big Bang Fair
One often invisible element of the Big Bang Fair is its evaluation. But why bother? Senior Researcher Tom May talks about how and why we evaluate.
It's important to evaluate the impact of what we do, to make sure that we are having the effect we want. Evaluating the impact of the Big Bang Fair helps us to understand if we have a positive effect on the young people who attend it and it helps us improve it each year by talking directly to the people it’s for. Through evaluation we are able to find out if they are more interested in science, technology, engineering or maths (STEM), to study or as a career, after the Fair than if they hadn’t attended. It also means we can show all those who have supported us the difference they have helped to make.
If we want to find out what young people think of The Big Bang Fair, or about studying or working in STEM-related careers, we somehow need to talk to them. The trick is to ensure that what the people you talk to say is similar to what those you don’t talk to would have said. Or to put it another way, that the people sampled are representative of the people that attended.
We do this by making sure that we talk to enough people and the selection of people we talk to isn’t biased. We can also compare the characteristics of those we have surveyed with those attending the Fair (or nationally). Using quotas is one way of doing this, so that we have enough people of each age and the proportions of boys and girls are balanced.
How do we evaluate?
Over 70,000 people are expected to attend the Fair each year. As it’s impossible for us to talk to (i.e. interview) all the people we need to, we do it through a short survey that can easily be done on a mobile phone or tablet.
Then, with the aid of statistical software (and a lot of coffee!), the analyst can begin to measure the impact.
We want to find out if attending the Big Bang Fair makes young people more interested in STEM. In theory, the ideal way is to run an experimental trial, take two randomly selected groups, send one to the Fair and one not, and compare them afterwards. Trials have been around for a while, there’s even one recorded in the Bible where the health of those on a meat or vegetable diets were compared (veggies won!). This is robust but also highly impractical – we can’t randomly make people attend or not attend.
Another option is to survey the young people attending the Fair before and after, and then compare the difference. However, because most register in classes or year groups, we don’t know who they are before they arrive. Talking to them beforehand is possible – we did it last year – but it is difficult and costly to do in large numbers. So we need a different approach. One that is practical but still robust.
So what do we actually do then? Each year EngineeringUK runs a national survey which includes young people. By comparing responses, we can show if overall they are the same as those who were surveyed before last year’s Fair. If they are, we can use these responses as a proxy for the pre-event survey. Then, we can survey the young people at and after the Fair. This will show if positive perceptions peak with the ‘buzz’ of being at the Fair and if they are higher afterwards than those before the Fair. We can then tell how well the Fair has ‘worked’.< Back to blog