Last week I wrote about the process and reasons for adding data items to the annual return. This week I am writing about the opposite – removing data items – and the very good reason(s) why.
The Pentecostal Church of Canada (PAOC) has just released its data request for last year. It is shorter. Much shorter. How much shorter? Well, the 2022 form (and its precursors) stretched to 7 pages and asked 81 questions. The 2023 form is 2 pages and has 12 questions. That is a radical shift in the data gathering philosophy. I will say more about the data items below.
The ACC has made some changes to our data collection in the last few years, cutting two data areas from our form – but nothing on the scale. Our current data set is based on a survey from 2012 when we established what dioceses collected and what we might want to collect. In the 2000s we were collecting, or rather trying to collect a much larger data set. We were largely failing to get the returns we were looking for.
And that points us to the main reason that “less is more” when it comes to data collection. Fewer questions means that respondents are more likely to respond. More data forms are likely to be returned complete and the overall picture from the data will be more complete.
Of course the number of questions in a form is not the only factor in getting a good completion rate, but it is a key factor. When the ACC coupled appropriate questions with a good return mechanism and a person who was willing to nag the dioceses in order to get the returns, then we saw the data that we needed.
I know from correspondence with the PAOC that response rate is the motivating factor in the dramatic reduction of the data set. It should be noted that their church system requires pastors to make a return, at risk of not having their ministry credentials renewed! But even that requirement doesn’t get them the return rate they want. I very much hope that the reduction in data set will have the hoped for response.
Sacrificing items
There is another reason for not collecting data items, which may form a rationale for items which are sacrificed in order to get a better return rate. Some items are simply more important than others. That may be for a variety of reasons:
In the ACC
The simplest items to sacrifice are those for which there is no imperative. In 2016 we chose to discontinue collecting data on justice and servant ministries in the ACC. Nobody was asking us about this data and the responses form the dioceses demonstrated a wide variety of understandings of the data required. In other words we didn’t know what we were collecting or why.
In 2020 we stopped collecting data on those “engaged in formation and learning”. We would not argue that this is not significant, but again there was a wide variety of understandings of the number in the various dioceses. And of course the data from 2020 was affected by Covid!
Read it all at NumbersMatter
