BLANKET SURVEY
I hate blanket surveys. This is not the research that looks into insulation and hot water bottles. It is the type of questionnaire that is distributed to everyone. Another name for this is saturation surveying.
I received one in my email box last week. It was from the HR dept of my employer (a big University). It was sent to all members of staff (I think and I’m sure they think so too – we have more than 800 teaching staff supported by up to 800 visiting lecturers). I answered another recently – the Staff Opinion Survey – and another the Staff Engagement Survey.
Every student at my University studies modules, let’s say six per year. Every module has a questionnaire and every student is expected to complete this module survey. That is 20,000 students being asked to answer 360,000 questionnaires over their degree. That is just the tip of the iceberg, there are other questionnaires: the National Student Survey (NSS), Library survey, the Student Barometer, the International Student Barometer and goodness knows what else.
After awhile the reflexes to refuse, delete and decline become second nature. Those who acquiesce will soon find themselves answering all the answers down the middle, perhaps one or two from left and right – this is called satisficing.
I’ve heard people justify this as “an opportunity for everyone to express their opinion” and “we need big enough numbers to be valid”. I have also heard, “this is a census - we need to hear from everybody”. It is NEVER a census. I think it is lazy and short-sighted. The only justification that I accept is for the NSS – because the results need to be broken down into small cells to allow sub-set analysis and course choice to be justifiable. But even then response rates are poor for some courses.
You may have guessed that I am very much against blanket surveys - I feel they reduce response rates in the long term and create an unbalanced final outcome. That unbalanced self-selected sample is sometimes weighted – but there is always a price paid when weighting is applied. Sometimes data collection is followed by sifting, effectively rejecting data records for some respondents who “mess up the results” – a sort of weighting that deletes records. Sifting basically means that we have wasted many hours of willing respondents.
Because we are asking more people to help than we need, we are wasting their time; we ultimately pay the price in response rate decreases in the future. And also by poor quality responses because of satisficing by respondents attracted by an incentive or simply wanting to help.
All of these blanket surveys should be banned and good old sampling used. Once selected these sampled individuals should be treated as VIPs – they hold the key to the truth. Instead we treat everybody with a lack of respect, polluting their lives with clutter and insincere requests.
Let’s take that University research. We need to stratify the staff by the most appropriate way for the study objectives (may be by department or not). We need to sample students in a way that ensures they are less likely to be selected for two or more modules.
We need to treat respondents like Very Important People. Look at all of these surveys to see how we are saturating students with surveys CLICK HERE
