I recently heard this quote in episode four of the excellent PBS documentary ‘The Vietnam War’, currently streaming on SBS on Demand. And it reminded me of the importance of deciding how one intends to use data before starting data collection.
Recently I have been involved in designing and automating a number of infrastructure audits. It is all too easy to audit what is most easily assessed and then draw conclusions from the results returned. But that may be putting the cart before the horse. Generally, it will be more fruitful to develop a model of the world and then look at collecting the data needed to populate that model. Doing it the other way around risk ‘making important what we have counted’ rather than ‘counting what is important’.
Hence, here is my suggested step for surveys and audits:
- Create a Model: Design and specify the algorithms and data analysis that will be used to consume the data. Run it with some test data or a small sample of real data.
- Design the Database: Design the scheme that will be used to store the data, defining data domains. This will prevent data errors.
- Design the Data Collection Methods: Digital forms will prevent data entry errors and speed data collection. I have been using FastField Mobile Forms recently. But that is by no means the only solution.
- Collect Data: Get out there.
- Analyse and Report: Run the model and present the results
One of the advantages of this approach is that it will be possible to analyse the data as it is collected. Systematic errors or erroneous assumptions in the model will be picked up early, allowing corrective actions to be taken.
Generally, 20% of time and budget should be spent on planning and design. That may seem a lot, but failure to invest this effort at the start of the project is the single biggest reason for projects not delivering the desired results. Furthermore, the model and database will have to be created anyway. And often doing this work retrospectively will be more difficult and time-consuming.
Surveys and audits are about data. This makes them an IT project. And that is something that is often forgotten. You will need a heart surgeon to perform heart surgery – but designing a data collection methodology, a database to store information and front end to analyse and present post-surgery health outcomes is an IT project that requires an IT background not a degree in medicine.