Invitation to Tender for Product Effectiveness Evaluation

Download this invitation as pdf here.

INTRODUCTION

Comic Relief invites proposals from suitably experienced organisations and/or individuals to define and undertake an evaluation of the digital products resulting from the Innovation Labs initiative. This partnership between Comic Relief, Nominet Trust and Right Here (Paul Hamlyn Foundation and the Mental Health Foundation) aims to utilise digital technology in the Third Sector to improve young people’s mental health and wellbeing.

Funding and developmental support has been awarded to seven not-for-profit organisations to build apps and websites, based upon briefs developed with young people aged 16-25, technology experts, youth workers and mental health professionals.

The evaluation will focus in-depth on three products which look to be successful in terms of their usage, their outcomes and their potential sustainability. We are looking for proposals which can evidence why (and how) the products are valued by users, the extent to which the products are achieving their intended outcomes and in turn, whether they are improving young people’s mental health and wellbeing (whether directly or indirectly).

The products will launch from October 2013 to February 2014, with funding finishing in June 2014. The evaluation will take place in 2014, with the timescale to be defined by the evaluator and has a budget of £20,000 to £30,000 (to include VAT and expenses).

The deadline for submitting a proposal (maximum 10 pages) is Monday 9^th December at 12pm. Please also inform us of your intention to submit a proposal by Friday 22^nd November. Further details on the Innovation Labs initiative and the seven products can be found at www.innovationlabs.org.uk.

BACKGROUND TO THE PROGRAMME

The Innovation Labs initiative began with a seven-month idea-generation process, delivered by a partnership led by Cernis and steered by a project team made up of the three funders and young people. About 200 ideas were generated through two face to face Innovation Labs events and a three-month online incubation period, narrowing this down to a set of eight final ideas.

A grants programme awarded seven not-for-profit organisations (ranging from small social enterprises to national charities) with around £47,000 each of funding to each develop one of the ideas in to a fully functioning product by June 2014. For some, their products will be integrated into existing services (for instance a micro-site within their website), whilst others will be developing an entirely new product (such as a smartphone App). The products and developers are:

• Doc Ready, developed in a partnership between Neontribe, Social Spider, Future Gov and Enabled by Design: An app to help young people feel more confident and get better results when they see their GP about a mental health issue. This launched in October 2013.
• Where to go for, developed by Sussex Central YMCA: A website to help young people find mental health support in their area.
• Madly in Love, developed by Youth Net: A platform where young people can share stories about how mental health affects their romantic relationships, get advice and feel supported.
• Minds Eye, developed by Mind Apples: An app that helps young people understand what contributes to mental health by linking their mood to moments in time.
• Keep the Trust, developed by Sixteen25: An online resource for non-mental health professionals who are working with young people with mental health problems.
• Mini me, developed by FACT: A personal interactive recovery guide inside a smartphone app. Young people with mental health issues can use it to avert or deal with rapid deterioration in their mental health.
• Medfacts, developed by Young Minds: A website that will give young people accessible, down-to-earth information on mental health medications.

The products are still in the development phase and we anticipate all to have launched a minimum viable product by February 2014. We have also commissioned a developmental evaluator to provide practical support for the projects. At this stage the focus of this support is to push for a product launch, alongside future planning to take the product to market and develop a sustainable business model.

The evaluation should bear in mind in that each product is designed for different audiences which include professionals (such as GPs, clinicians, youth workers) as well as young people, with some products having dual beneficiaries.

The young people using the products will also have different levels of wellbeing, with some products targeting any young person, others targeting those with mental ill-health who are at different stages of diagnosis or help-seeking. In turn, whilst the products are seeking to improve young people’s mental health and wellbeing overall, their intended outcomes largely involve interim steps towards this, such as empowering young people to make informed decisions, to seek help and support, to take ownership over their recovery.

BACKGROUND TO THE EVALUATION

We are looking to commission an evaluator or organisation that is able to demonstrate a sound approach to evaluating the value of digital products, as well as relating this to young people’s mental health and wellbeing outcomes. We anticipate that the budget and scope of the evaluation will focus in-depth on three products. The final selection of these products will be made with the evaluator when they are appointed, in consultation with the projects, the developmental evaluator and the funders. We assume these products will be the most successful in terms of their adoption and potential sustainability and it is likely that one will be Doc Ready, the first of the products to launch. However, we are open to any proposal which involves evaluating the products to a greater or lesser extent. Overall, the proposal should provide a compelling case as to how the approach taken will achieve the evaluation purposes and be appropriate for the context (products and users).

Evaluation Purposes Individual Products

1. User value: Do the intended users choose to engage with the product, why do they value it and does this meet their mental health support needs?
2. Intended value: Is the product is achieving its intended outcomes? These products add value to existing services, products and sources of support for young people (rather than replacing them). As such, their intended direct benefits to users should be evaluated, whether this is about improving confidence to access a service or helping the service to access beneficiaries.
3. Social value: Is the product improving young people’s mental health or wellbeing? These products should be helping to improve young people’s mental health outcomes as an end result, whether directly, such as encouraging positive behaviours or indirectly, such as empowering young people to take ownership of their support needs.

In order to address these questions the evaluator will be able to draw upon existing monitoring and feedback data collected by products and can add an additional layer of analysis. We are also looking for the evaluator to explain how they will collect primary data in order to provide robust evidence of the intended and social value of products.

Our Evaluation Objectives

We are commissioning this evaluation, alongside other learning and dissemination activities, in order to achieve the following:

• Provide funders and stakeholders with evidence of whether the Innovation Labs initiative resulted in digital products which are used and achieving their intended outcomes
• Give project teams’ insight in to their product effectiveness and provide recommendations about how they could progress the user, social and market value of their product
• Convince next stage partners and investors of the current and potential impact of these products and how they might benefit from being involved
• Encourage other funders and third sector organisations to engage with digital technologies to achieve social change

EVALUATION DELIVERY AND TENDER PROCESS

We are looking for interested parties to submit a proposal for the approach they believe is best to achieve the evaluation purpose, in light of the background to the programme and the products. For this reason, we have specified a range for the budget, which we expect to cost between £20,000 to £30,000 (to be inclusive of VAT and expenses) and for this to be appropriate to the evaluation activities involved. We expect an individual report for each of the products evaluated in depth, as well as an overall evaluation report which brings these together.

The proposal should be a maximum of 10 pages and outline the approach taken and rationale, the practical research activities involved, a clear timeline and a budget breakdown. Given the choice of the final three products has yet to be defined, the proposal should outline the approach and types of methods they expect to employ, with the expectation that the proposal can be outlined in full once appointed. The proposal should be accompanied by short CVs (maximum 2 pages per evaluator), highlighting any relevant experience relating:

• Undertaking research with young people aged 16-25 (note that the evaluator must have the appropriate policies and procedures in place at the start of the evaluation).
• Developing digital products and/or evaluating their effectiveness
• Evaluating mental health and wellbeing outcomes

WHAT TO DO NEXT

Please inform us of your intention to submit a proposal by Friday 22^nd November 2013. The deadline to submit the full proposal is Monday 9^th December 2013 (12pm), with interviews scheduled for January 2014. The proposal and any enquires should be directed to Nissa Ramsay, Monitoring and Evaluation Manager, N.Ramsay@comicrelief.com (02078202734) and Billy Dann, Programme Manager, B.Dann@comicrelief.com (02078202727).