Consumers International’s (CI) Bloomberg Advocacy Incubator (BAI) project team commissioned CommsConsult to develop and design a Monitoring, Evaluation and Learning (MEL) System to help assess the impact of their contribution within the greater global and multi-faceted programme of support offered to GRSP on road safety advocacy.
Designing a monitoring and evaluation system for advocacy is a complex process that poses, often unique, challenges. However, through drawing on ‘complex’ M&E approaches, the challenges can be overcome.
Comms Consult’s intervention was specifically aimed at helping CI to better document and monitor impact around two different (but inter-related) elements of their advocacy support work: 1) capacity building and 2) policy impact.
CommsConsult carried out a series of interviews and conducted desk based reviews in order to fully understand the current dynamics between all of the different stakeholders (beneficiaries, partners and donor). From this we developed and situated a theory of change specific to CI’s role within the greater context of the BAI programme, that spoke to its current focus on road safety-related health issues and that clearly captured key assumptions. The theory of change was then developed and tested against the overarching MEL framework at three levels: output, outcome and impact. Identifying what should be captured and how, CommsConsult subsequently developed and delivered a suite of quantitative and qualitative tools relevant to the framework, thus, developing an MEL system that could both measure and explain impact at the levels articulated by the client. These tools included a capacity assessment tool (building on CI’s own internal working), a partner collaboration continuum, stories of change and policy tracking logs.
Final outputs were delivered in the form of a written report which could be presented to both partners and donors, and also in the form of a workshop, which introduced attendees to the tools outlined in the report, enabling greater uptake of CommsConsult’s recommendations going forward.
CommsConsult’s approach was based on the following principles:
- Participation: We worked with CI to help identify ‘what success looks like’ and what they would regard as suitable indicators showing both how capacity has been built, and what the results of these improved skills and knowledge sets were.
- Relevance: We aimed to understand and review any existing (explicit or implicit) MEL plans and tools that CI or its partners used, before proposing and adapting tools that would complement and strengthen these systems in the desired areas.
- User friendliness and efficiency: We developed an MEL system and processes that were complementary to the grantees’ and end user’s own MEL systems so that collection and sense making was easier. Without this, the systems and processes -however smart – would not be sustained.
- Adaptable and context-specific: CommsConsult ensured that the system delivered worked on the two required levels: both at a generic and transferable level for CI and at a context-specific level for the particular project and instance.
- Fit for purpose: CommsConsult ensured that the proposed tools are able to generate information that can be translated and communicated to key stakeholder groups by staff who understand and are proficient in their use.
