Learning has been a critical part of the Innovation Labs journey. When you read the posts and stories of how these apps and websites have come into being, you can see how much the folks involved have learnt along the way. Brilliantly for us – they’ve shared it through their own, and this website’s, blogs.
But as well as this, the Labs initiative has invested in more formal evaluations – three of them all together. Here, people external to the project have been asked to evaluate different parts of the process.
The first was a report on the co-creation and product funding phase.
The second, reporting later this year, is a very practical hands-on developmental evaluation of the production phase.
What Will Mindtech Evaluate?
Although we know lots about how these digital tools have been developed – until we know more about how people are using them, we won’t know if they are actually doing the good work intended. In our evaluation, we are mainly interested in how young people are using the apps and websites and what difference using them makes to their mental health and wellbeing. To do this we plan to use a range of different methods.
Method 1: Analytics
The fantastic thing about digital is that we can find out an awful lot about how people use the tools from the information that is the automatically created when they do (often called analytics). We will be able to discover, for example, how many people use the tool, when they use it, for how long and which parts they use the most. People won’t need to register or sign up for this as we won’t be finding out any personal details – it’ll be all numbers and statistics. Although anonymous, this data will be really helpful and it’s a great bonus for us researchers. It’s a lot harder to come by this type of data in non-digital health services!
Method 2: Existing Evidence
We’ll have all the information and materials that have been created about the tools while they’ve been in development, for example, the blogs, project reports, prototypes, social media campaigns and even some short films already made about some of the tools.
Method 3: Direct Engagement
But thirdly, and most importantly, we’ll need to learn directly from young people who are actually using the tools.
We’re planning to give different options for people to participate in the research directly and we’re working with some of the original young ‘Labbers’ to develop these approaches (its really great to have their support).
For those that like filling in questionnaires, that will be one option. We’re not sure many people do, so there’ll be a prize draw as encouragement.
For those that like talking about their experiences, there will be an opportunity to take part in an interview with one of the research team. We’ll be able to do this via Skype or telephone so people all over the country can take part. We are also exploring ways people can upload secure videos so they can send us a video diary of their feedback.
When Will We Have Results?
We plan to collect all this information together between June – November 2014. Then it’ll take us a few month months to work out what it’s telling us about the use and impact of the websites and apps.
We’ll be reporting back to Comic Relief and the wider Innovation Labs team in early 2015.
Along the way, we hope to capture people’s experiences on film – so look out for some short films on the website too.
Who Are We Anyway?
The MindTech Healthcare Technology Cooperative is funded by the research arm of the NHS, the National Institute for Health Research. Our role is to encourage new technologies for mental health. We want user and clinician experiences to inform the need for and design of new technologies and support the greater uptake of technology in the NHS.
We bring lots of experience of health services and technology research. Working with Innovation Labs is a great opportunity for us to be part of the movement working out the best ways to develop and evaluate digital healthcare.
Better get our skates on as there lots to do!