Valid International
In late June this year, ENN interviewed Kate Sadler, Director of Programmes at Valid International (VI), and followed up by speaking to Steve Collins, one of the founders of VI and still at the helm. Kate is an old hand herself, having first joined VI as a nutrition advisor in 2001 (after meeting Steve at a UNICEF meeting in Ethiopia). She spent six years working on Community Therapeutic Care (CTC)/ Community-based Management of Acute Malnutrition (CMAM), during which she completed her doctorate. After a five-year break in the US (as Assistant Professor of Nutrition at the Feinstein International Center (FIC), Tufts University), she returned as Programmes Director, and now oversees all projects and programmes as well as supporting the strategic direction of the organisation.
We began by revisiting the profile ENN did on Valid back in 2006, when Steve Collins and VI co-founder Alistair Hallam were interviewed[1]. Steve had described how he wanted to keep the Valid ‘vibe’ fairly informal. Kate laughed and said that the informal vibe is still there, although they have “brought in a few structures and systems” to professionalise since then. Ever the visionaries, Steve and Al also saw “the potential for more experienced staff to ‘bud off’ and form groups which they manage more autonomously.” Twelve years on has seen the creation of Valid Nutrition (VN)[2], which is focused on the food product area, and Valid Evaluations[3] (VE), the most recent offshoot led by Alistair Hallam. Before we went any further, Kate explained the differences. They are all separate entities: VI is a social for-profit enterprise based in the UK; VN is an Irish-registered charity; and VE is a separate, UK-based entity.
VN’s work centres on local production and research and development of ready-to-use food (RUF) and related research, such as increasing the use of locally sourced ingredients and the impact this can have on food security and lowering costs. VI and VN have a good, collaborative arrangement, sometimes sharing “people resources”, while VN sometimes draws on VI research capacity. They submit joint proposals where there is common ground, such as testing the efficacy of novel, lower-cost product formulations made from locally grown ingredients and their impact on severe acute malnutrition (SAM) treatment and prevention. VE expertise lies in assessment and (as the name suggests) evaluations. This complements VI’s work, which involves primary research and innovations around indicators and methodologies (a significant workstream within VI called Measures), and again VI and VE work very collaboratively here.
So, who is at the helm these days? Steve is at the managerial apex as sole director and works from home in West Cork, Ireland. A team of ten core contracted staff covers administration, finance, programme management and technical research, enabling the Measures work programme. Most of the team are based in the Oxford office. VI also draws on a pool of consultants worldwide to deliver projects, including six core consultants (associates) who regularly work with VI and help leverage work at country level[4].
I asked Kate what was different about Valid now compared to 11 years ago, when we last ran their profile in Field Exchange. She explained that, back then, community therapeutic care (CTC), the precursor to the community-based management of acute malnutrition (CMAM) approach, was the sole focus of VI – implementing research and providing technical support to implementers or governments to pilot and subsequently scale up CTC/CMAM. As this evolved, various offshoots of work developed, such as exploring how the approach fits with food-by-prescription services for HIV and what makes a programme successful from a community perspective. As CMAM became more accepted and mainstream as an approach and its capacity to deliver was strengthened, the demand for this type of technical support began to reduce. Increasingly, requests came in to help agencies evaluate what they were doing and to measure aspects like coverage.
“As part of CTC’s research and development we realised we needed to better assess the population-level impact of programmes. There was a particular gap in understanding how well programmes were doing in reaching the whole of an affected population – we were able to assess numbers treated and recovered, but traditional tools couldn’t give a direct estimate of programme reach or coverage.”
That question led to VI’s collaboration with Mark Myatt at Brixton Health and the start of the development of assessment methods that could better assess coverage of and access to selective feeding programmes. This became a growth area for VI and now forms a big stream of work under the Valid Measures team. The work focuses on the development of more efficient and effective assessment methodologies to assess coverage and meet the assessment needs of many different types of intervention around child survival and prevention actions. For example, VI has recently been examining coverage of urban water, sanitation and hygiene (WASH) programmes in Bangladesh and regularly applies new methods to aid rapid assessment in different sectors, as well as monitoring and impact evaluation.
In their work around surveys and assessment, VI works with and draws on methods from many other sectors. Kate explained that VI is engaged in four main elements. First, the metrics; i.e. what and how to measure, indicator development and sample design. This work has included development and testing of easy-to-use infant and young child feeding (IYCF) indicators, work on more complex, composite-type indicators to measure poverty and resilience, and the application of spatial sampling techniques to improve usability of assessment outputs. The second area is the analytics; i.e. how to analyse the data, statistical algorithms and methods. VI is continually trying to adapt and improve survey methods and make them more cost-effective. Much of this is about reducing sample sizes, drawing from statistical methods (such as PROBIT and ‘bootstrapping’) applied in other sectors. The VI approach is driven by generating information that is useful for programme and policy decision-making. To achieve this, surveys need to be more localised, with indicators mapped to the district level; this is where spatial sampling comes into its own. The third area, training, supports workshops, field training, manuals and online courses to help partner organisations build capacity in the area of assessment. The fourth area, technology, supports the systems and platforms for data capture and processing.
I asked Kate what common challenges they come across in evaluations. She was quick to pinpoint the need for agencies to be able to identify causality:
“Nutrition is so multifaceted – it is very difficult to figure out whether an intervention has caused any change or not. Everyone has ‘theories of change’ these days, but it’s hard to design assessments that are able to cope with the realities of programming in country contexts and also deal with the issue of attribution. At one end we have good academic studies. while at the other end we deal with the more anecdotal perspective of implementers.”
Kate described how VI is often asked to design a baseline and endline study with no consideration of capturing what happens in between: “What we should be doing is identifying the changes along the way and figuring out how we measure as we go along to inform programming at the time”. Indicators are frequently a challenge, often measured in different ways and quite sector-specific. Also, standard indicators aren’t necessarily the most useful, context-specific indicators. VI is sometimes brought in at the design phase to decide on the most relevant indicators; this in turn can generate more questions (and work). Whether to use mid-upper arm circumference (MUAC), for example, came up as a recent question for work in India. In this instance, VI helped make the case for including it in assessments. Kate also pointed out that assessments have not kept up with the demands of multi-sector programming and that agencies are struggling with how to bring it all together. VI is increasingly asked to help with this challenge.
So, who funds VI? It is typically contracted by United Nations (UN) agencies and non-governmental organisations (NGOs) on specific pieces of work, sometimes responding to grant calls as part of consortia and often “partnering with like-minded agencies that provide a platform to allow us to do what we want to do”. VI is not directly contracted by government, but often works with it. There is usually some capacity-building component; for example, data collection teams often come from the national statistics bureaus. Currently VI is undertaking research in Ethiopia in partnership with government, which was a requirement of the funder (the Bill & Melinda Gates Foundation).
Back in the early days of VI, generating evidence was critical to CTC/CMAM success and operational research remains central to VI. This workstream is overseen by Paluku Bahwere, the VI research lead. Kate mentioned a key piece of work VI is currently involved with: a Gates-funded project to look at the implications of changing the MUAC admission cut-off to the new World Health Organization (WHO) recommendation in terms of impact on CMAM programmes and cost-effectiveness. Also in Ethiopia, VI has led the design, implementation and data analysis for the impact evaluation of the ENGINE project[5], as well as two other research studies. One is looking at nutrition and health-related outcomes of children treated for SAM in a community-based programme in Jimma, Ethiopia; the other is a prospective cohort study from a food-secure setting in rural Ethiopia on the treatment of moderate acute malnutrition (MAM) in children, summarised in Field Exchange 54[6].
As some of our readers may remember, the early days of CTC/CMAM were steeped in debate, conflict and controversy, with opponents of this new, community-based approach arguing that it would contribute to child deaths. VI’s commitment to generating evidence therefore had to be strong and steady. I was keen to know what Kate saw as the next big question for the organisation. She responded by saying that the team has developed ideas around the prevention of stunting through market-based interventions, including targeting consumers with low-cost, highly fortified supplements. “Many are talking about harnessing the power of the market to address undernutrition”, she explained, “but the question is, will a market-based intervention work and what kind of impacts on nutrition to expect?” In order to answer this critical question, VI is proposing and currently seeking funding for a five-year research programme.
Asked what makes VI unique in its assessment and survey work, Kate checked with her colleague, Ernest Guevarra, who leads the Valid Measures stream of work, for his insights, given his on-the-ground experiences of assessment. He reflected on the niche that VI occupies:
“We occupy a space between those organisations implementing more standard assessments (they have nutrition/health sectoral expertise but not the stats/spatial epi/technology to do the assessment/indicator design or innovation) and big developers such as Nielsen, who focus uniquely on development of new surveys – they have the stats/spatial epi/technology to do assessment/indicator design and innovation but not the sectoral expertise. VI has high-level expertise in health/nutrition as well as the technical know-how to develop assessment methods/indicators to address very specific sectoral needs. This places us in a unique position to support understanding of assessment needs and the skill set to meet these needs in the most efficient and useful way possible.”
Having finally managed to track Steve Collins down at an airport as he transited to a flight to Myanmar, I asked him the ‘big’ question: “What is your vision for the future of VI?” He was pretty clear on this: “The drive to generate quality data to provide evidence on what we all do, and hold ourselves to account, remains the over-riding guiding principle of VI’s work. VI looks to push the boundaries of what we can do.” He pointed out that the fact that the core team – Kate, Paluku, Ernest and others – has been with VI from the early days has meant it has remained true to this core principle while broadening technical horizons.
I asked Steve what the main challenges over the last ten years have been for them. One has been the transition from being a “small band of rebels” promoting an alternative approach to an expert group in what became a mainstream technique; VI had to work on changing people’s understanding of what VI was about. It has also had to up its game in what Steve described as “customer service”. In the old days, work came to VI and it was very much focused on doing the heavy lifting on the ground. Now they undertake work with clients where there is much more emphasis on communication and a professionalisation of this interface. He went on to say how communication within VI among the staff, as well as externally, has become even more critical as they have expanded (and sometimes contracted).
Another interesting challenge has been around intellectual property. VI has always had an open-door policy when it comes to sharing developments. “We made early RUTF [ready-to-use therapeutic food] patents freely available, we trained people up to do CTC [community-therapeutic care] (and did ourselves out of a job in doing so), we developed methods like CSAS and SQUEAC and trained users”. However, as VI developments were taken on board by agencies, VI often got cut out. Steve reflected that “if you give away intellectual property, it is very difficult to have a viable business model”. Today VI is happy to pass on expertise and knowledge around all its assessment work, but tries to do so through structured partnerships with agencies. Another challenge, common with small organisations, is managing the capacity to deliver with the jobs on hand (supply and demand). It is quite a balancing act. In the early days, when CTC/CMAM work was going strong, VI had people on long-term contracts. Nowadays, shorter-terms contracts are the norm.
I asked Steve about how the developments of VN and VE sat with his original vision. He reminded me how, in the original CTC model, local production and delivery of RUTF was “one of the circles” and a fundamental part of the approach. To develop this strand of work, it made great sense for VN to form and focus on the “route to market in more depth”, with VI remaining focused on the research/technical aspects. Similarly, VI was increasingly undertaking evaluations and it made good sense for more formal/commissioned evaluations to branch off as a separate entity. As Kate reflected, there is strong collaboration between the Valid offshoots; Steve sits on the board of VN and so is involved at a strategic level, rather than day-to-day. Similarly, his role in VI is much more hands-off and “peripheral” than in the first flush of VI, an evolution which has been a conscious decision on his part. He described how “nutrition is plagued by old experts who are often conservative and don’t run with the times”. He is no longer on the ground like he used to be, aside from a few “pet projects” such as his work with government in India. “The vitality and responsiveness to issues need to be directed by people on the ground doing the work”; he is no longer there, but the VI team is – a group of individuals in whom he has the utmost faith to drive VI forward. He sees this “transfer of power to ground level” as critical to maintaining VI’s relevance.
Steve concluded, “In a nutshell, we are about evidence-based development and relief. We need accountability of the system – do we have effective impact? There are only limited pots of money available. We need to make what we have got work”.
ENN’s office is next door to VI in Oxford and I should confess that I have only had a limited knowledge of the rich web of cohesive work VI continues to spearhead, in our sector and in others. As ever with success stories, the individuals involved ‘make or break’ small organisations and it is clear that the longstanding commitment of many individuals at VI has been critical to its longevity and relevance. Being able to innovate and adapt to changing contexts is no easy task and challenging on many fronts; VI has done that, and more. We’ll be keeping a closer eye on what they’re up to – and sharing it through Field Exchange – from now on.
[1] Jeremy Shoham (2004). Valid International Ltd. Field Exchange 23, November 2004. p22. www.ennonline.net/fex/23/agencyprofile
[2] http://www.validnutrition.org
[3] http://www.validevaluations.org