Big data seems to have become the latest must-have for many researchers. With large data-sets which can be analysed computationally to reveal patterns and trends, it’s easy to be seduced by the opportunities it can offer. However, data is only relevant if it has value to a user, and there are often limitations which you need to be aware of when embarking on a big data project. Here we get to the truth of working with such data-sets by drawing on experiences from the Cultural Landscapes project.

In 2016, Arts Connect commissioned BCMCR to undertake the research and development of a methodology to better understand arts and cultural provision for children and young people in the West Midlands. The project saw the successful creation of a tool which could be used for collecting, analysing and sharing relevant data, which was shown to offer a significant opportunity in gaining a better understanding of current engagement and of increasing impact. However, the project also illustrated considerations of working with big data, particularly at a research and development stage.

Big data requires commitments – Working with big data requires all parties involved to be committed to the project, it can mean a considerable investment of time and effort. Highlighting the potential benefits and opportunities of collecting and sharing data can aid this process along with a lead organisation having the social capital required to ensure the participation of partners.

Resources and supporting partners – Beyond the commitment needed to work with data is the need also to manage change and the resources required. Data-driven tools can be new to many, meaning they may feel unequipped, which can lead to resistance. New practices could, therefore, have implications for staff time, skills, confidence and require additional training.

Value data over the tool – It’s easy to get distracted by data-driven tools, but it’s important to remember that its value lies in the production of data which is relevant and of value to users. This requires the feeding of robust and updated data from sources. Recognising what the tool can and can’t do is also key to avoiding the disappointment of partners, as is presenting findings in an accessible manner which doesn’t require a high level of technical data literacy

Technical constraints (and working with a budget) – While big data and the tools required to work with it are becoming more and more accessible, there are still considerations to be made, particularly if your project is at the research and development stage, or working within a tight budget. It might be that while free software can be used for an R&D project, a custom-built tool might be required to enable it to be deployed professionally.

While it’s important not to over-celebrate big data and what it can offer, neither should the limitations be seen as a reason not to proceed. They are better thought of as considerations which need to be taken into account when planning a project and formulating an evaluation plan. M&E Lab provides a resource toolkit for the planning, recording and analysing of project evaluation, which can be accessed here.