Operational research in low-income countries: what, why, and how?

Summary of research¹

A Lancet published article puts forward a definition of operational research, articulates its relevance to infectious-disease-control programmes, and describe some of the enabling factors and challenges for its integration into programme settings and into changing policy and practice.

From a health programme perspective, a pragmatic definition of operational research is the search for knowledge on interventions, strategies, or tools that can enhance the quality, effectiveness, or coverage of programmes in which the research is being done.

What is operational research?

Operational research involves three main types of method: descriptive (cross-sectional, if a strong analytic component is also present), case-control, and retrospective or prospective cohort analysis. Basic science research and randomised controlled trials should not be included as operational research. The randomised controlled trial determines efficacy of an intervention in a strictly controlled environment with inclusion and exclusion criteria, whereas operational research should assess effectiveness within routine settings. Both types of research play an important part in the generation of new knowledge: the randomised trial provides clear-cut data on the efficacy of an intervention in defined groups of patients, whereas operational research determines how such interventions are translated into benefit in the heterogeneous setting of routine care.

The key elements of operational research are that the research questions are generated by identifying the constraints and challenges encountered during the implementation of programme activities (prevention, care, or treatment), and the answers provided to these questions should have direct, practical relevance to solving problems and improving health-care delivery. A strong connection exists between good monitoring and evaluation of infectious-disease programmes and operational research.

Good quality data on cases and treatment outcomes can be used to do operational research, which in turn can help to improve the routine data collected in the field. Nothing is more encouraging to healthcare workers than to see their work in recording and monitoring data on treatment cards and registers being used to answer important questions, provided that this performance is recognised and applauded.

There are at least three reasons why operational research is relevant to health. To improve programme outcomes in relation to medical care or prevention, to assess the feasibility of new strategies or interventions in specific settings or populations, and to advocate for policy change.

The study question must be of importance to programme implementation for ‘buy in’. Much of the internationally published research done in Africa has been generated by academic institutions and researchers, predominately reflecting their interests or based around basic science or questions of intervention efficacy. Although useful, this type of research needs to be balanced by increasing the work done by operational organisations (e.g. non-governmental organisations (NGOs)) that will have different perspectives. Different actors will naturally have comparative advantages for particular kinds of research that can benefit programmes. For example, an academic institution might be best placed to design and implement a randomised clinical trial or a vaccine study, whereas an implementing organisation might be best suited to take the lead in feasibility and acceptability studies. If research is disconnected from health-service delivery with little or no input from programme staff in the research design and process it may be being resented as an additional and often unwanted extra on existing busy and often overburdened services.

Enabling factors and challenges

Factors that enable operational research and its translation into policy and practice include:

• Research questions are generated from within programmes
• Research planning, agenda setting, objectives, targets, and budgeting are included within programme plans and as agenda items in programme management meetings
• Research projects use simple designs and are focused to answer implementers’ questions
• Close collaboration and partnership has been established between researchers and programme managers
• Research is done within existing systems and not done in parallel
• A competent research officer works alongside the programme manager
• Training, mentorship, and on-the-job supervision is sustained over time
• Sufficient programme capacity exists to host workshops, present, and discuss research findings, and ensure their translation into policy and practice
• Programme staff have access to scientific literature through subscribed journals or the internet
• Sufficient numbers of programme staff are available with the capacity to do operational research, write up manuscripts, and publish relevant research
• Funding for applied research is available and individuals develop a desire to participate in research and are mentored
• NGOs and other stakeholders are recognised and have a contributory role in operational research
• Good quality, appropriate, and relevant research gets translated into policy and practice and thereby has a spin-off effect to stimulate more research

One challenge is that foreign academic institutions often have the funding, time, and mandate for research and thus the associated power in decisions about what gets done. Local institutions should also be supported with money and staff for operational research, thus allowing them the necessary independence to make decisions, take responsibility, and establish partnerships that are more equal in resources and decision-making power.

The authors suggest building a research agenda into district and national programmes, based on local needs, but primarily reflecting the research priorities of the country. Within a country, it is important to have a coordination mechanism to provide a clear strategy of who sets research priorities and how choices are made at national level.

A bibliographic analysis of tuberculosis research done globally between 1997 and 2006 showed that Africa, which has the highest tuberculosis case rate burden in the world, contributed only 7% of global research output.

Local programmes have a tendency to outsource research to academic institutions, which then set up parallel research systems or affiliated sites. Although this might be a highly efficient means to produce quality research and scientific publications, if there is no satisfactory mechanism for integration, collaboration, and communication with the programme, this approach might hinder the development of operational-research capacity by drawing national researchers away from national programmes. Furthermore, because research institutions and technical agencies (either international or national) normally have no mandate or responsibility for implementing research findings after their studies are completed, the results often end up being sent or presented to busy programme managers, who have no ownership of the research and who are therefore unlikely to direct the effort needed to translate the research into policy and practice.

A change to a partnership model would enable greater involvement, co-ownership, and responsibility of programme staff along with researchers and policy makers. For example, the research question should be developed by the entire team, including those involved with questionnaire development, collection and analysis of data, and dissemination of the results. Planning at this stage also requires a clear engagement with the people who make decisions on policy so that they are aware of what is being asked, supportive of the research, and interested to learn about the findings.

Capacity and time for research activities, such as writing study protocols or dealing with peer review are often lacking within most programme settings but are essential to see research to completion. However, if they are planned as an essential part of the programme, they can be accomplished (see Box 1 for an example).

The failure to publish research is not just confined to the low-income countries: a recent report found that only 53% of 79 research studies reported in conference abstracts were published in peer-reviewed journals after nine years. Common reasons for so-called research ‘waste’ include the wrong choice of research question, poorly designed studies, and failure to publish relevant research promptly or at all.

There is a perceived need to create a socalled critical mass of trained researchers within health programmes to ensure that sufficient numbers of researchers continue in post to sustain future research. The Japanese Research Institute of Tuberculosis, the Japanese Foundation for AIDS Prevention, the International Union Against Tuberculosis and Lung Disease, and the US Centres for Disease Control and Prevention are among some of the institutions that support international training on operational research.

There is also a need to assess whether current capacity-building initiatives are having an effect, such as tracking personnel after training (e.g. through databases) to document the outcomes, explore any barriers, and capture suggestions for improving the situation. Many researchers from low-income countries, even after obtaining PhDs, do not take up research when working back in disease-control programmes. This might be because they end up in senior-level management posts, the infrastructure to plan or do research is lacking, or there are simply no opportunities

Creating opportunities

There are various ways in which these opportunities could be created. First, small grants could be offered to pursue locally applied research. Second, junior and senior operational research fellowships could be created for colleagues in low-income countries with active mentoring by international researchers, institutions, or NGOs.

Bureaucracy should be kept to a minimum with the main focus on deliverable outcomes that would include publications with specific benefits to programmes and communities. Targets for research output should be set (e.g. one or two research papers each year submitted to a peer-reviewed journal), with financial and technical support continued when targets have been met and termination of support if targets have been missed. Young national researchers should have the opportunity to present scientific abstracts and participate at regional and international conferences, and more attention should be paid to teaching the principles of how to write scientific papers and to mentoring.

Attention must also be paid to the problem of poor access to up-to-date scientific literature, and despite laudable initiatives (such as the Health InterNetwork Access to Research Initiative), this remains a barrier in low-income countries. Free and open access for all articles of interest to low-income countries is urgently needed. For example, Médecins Sans Frontières (MSF) has negotiated with publishers to allow free access to all articles written by its staff.

The distinct role for NGOs in operational research should be recognised for two main reasons. First, NGOs such as MSF often work in conflict settings, with marginalised and vulnerable populations, or with neglected diseases. Academics rarely have access to such settings, and national programmes might decide they do not have sufficient resources to study them. Research in these areas is, nevertheless, needed to better understand how to manage questions such as mental health issues in war zones, treatment and diagnosis of neglected diseases, or offering of HIV/AIDS care in slum settings. Secondly, NGOs are, by mandate, implementers and can thus be involved in the translation of research findings into policy and practice. If they have skills in research and advocacy as well as sufficient financial and human resources, then they probably have the potential to actively engage in operational research and help change practice.

However, NGOs are sometimes not the appropriate entities for designing or implementing research. They might lack the institutional support, culture, and skills for interacting with national programmes and decision makers. NGO focus might be on solving localised, short-term problems, they might have had little exposure to systems thinking .and they might lack the training and capacity to do rigorous research. They might also have a rapid turnover of staff, which hinders the sustainability of research and the ability to build up trust and understanding with country partners. These points might explain why NGOs rarely undertake research, are rarely asked by country programmes to do so, and why the research they do undertake is sometimes badly done, with little or no programme impact.

The authors conclude that they have made the case for the importance of operational research as a necessary component of health programming in low-income countries. What is needed now is further development of operational- research capacity, allocation of specific resources, and the need for different participants such as international and national academic institutions, national programme managers, and NGOs to work together in promoting operational research.

¹Basinga. P et al (2011). Effect on maternal and child health services in Rwanda of payment to primary health-care providers for performance: an impact evaluation. Lancet 2011, 377: 1421-28

¹Zachariah et al (2009). Operational research in low-income countries: what, why, and how? The Lancet Infectious Diseases, Volume 9, Issue 11, Pages 711 - 717, November 2009