The International Initiative for Impact Evaluation (3ie) is an international NGO that promotes evidence-informed equitable, inclusive and sustainable development. We support the generation and effective use of high-quality evidence to inform decision-making and improve the lives of people living in poverty in low- and middle-income countries (L&MICs). We provide guidance and support to produce impact evaluations and associated formative evaluations; systematic reviews and syntheses, evidence gap maps, and to use that evidence in decision-making and capacity development. Our work encompasses a wide range of topics, sectors and themes, including but not limited to health, nutrition, education, agriculture, governance, microfinance, climate change, humanitarian interventions, and social protection. 3ie is registered as a non-governmental organization in the United States. It has offices in New Delhi, London and Washington, DC.
3ie is supporting the Millennium Challenge Corporation’s (MCC) Monitoring and Evaluation Division to create a decision tree of evaluation methodologies that can be used to evaluate policy and institutional reform (PIR) projects. PIR projects are particularly challenging as development interventions, both from a project design and M&E perspective. To start, it is often difficult to define and quantify institutional problems, which makes it difficult to precisely target and address key constraints. Traditional project logic models that are commonly used do not adequately account for the complexity associated with PIR projects. Similarly, there is insufficient clarity in the development field generally about the types of M&E methods best suited to adaptive projects, particularly if the aim is to establish causal impact (or the plausibility thereof). Both these factors make it difficult to attribute beneficiary-level outcomes to the intervention. To support this division in identifying appropriate methods for causal inference, 3ie will work collaboratively with the MCC PIR Working Group to develop a “menu” and/or decision tree of evaluation methodology and data collection methods for PIR.
We are currently seeking a Complex Evaluation Specialist Consultant to support an ongoing project with the MCC. This work will be managed by 3ie’s Washington, DC office but the consultant can work remotely.
The consultant will provide input and technical guidance on complex evaluations of policy and institutional reform in collaboration with 3ie staff and the MCC PIR Working Group throughout this activity.
The successful candidate will have significant causal attribution evaluation experience and will have demonstrated expertise implementing research and translating research evidence for decision-makers in L&MICs. We are seeking a highly diplomatic and entrepreneurial individual who is comfortable taking initiative and who thrives in a matrix-managed, fast-paced environment.
The consultant will report to the Director of the Washington, DC office and to the Lead Evaluation Specialist leading this work.
- Provide input and technical guidance on methods that can be used for the design, monitoring, and/or evaluation of complex and/or adaptive interventions, with particular relevance to policy and institutional reform. The selected methods should:
1) Directly or indirectly assess (or help elucidate) the causal relationship between the intervention and outcomes. This includes, for instance:
o Econometric and/or quasi-experimental designs that establish an appropriate identification strategy (e.g., fixed effects, natural experiments, etc.) o Non-econometric evaluation methods that may establish the plausibility of a causal relationship (e.g., process tracing, systems approaches)
o Simulating or modeling methods that may predict a causal impact
o Other research approaches that complement or enhance methods used for impact evaluation (e.g., by identifying key contextual factors, feedback loops, potential unintended consequences, client/beneficiary experiences and perceptions, program and economic costs, etc.)
2) Be relevant to one or more stages of a project lifecycle, including (but not necessarily limited to):
o Problem identification and project design
o Routing/ongoing monitoring and evaluation
o Impact evaluation
3) Include description of key use cases; data collection approaches and sources (including non-conventional data sources, such as big data); strengths, limitations, and other considerations; and reference materials.
- Contribute to developing search strategy and protocol for scoping review of PIR interventions
- Review and quality-assure deliverables, including decision tree/menu of monitoring, evaluation and data collection methods for PIR
- Participate in and contribute to team and partner meetings, including with 3ie and MCC staff; present key components of the work as needed
Qualifications, experience, and skills
Education and work experience
- Ph.D. in Economics, Political Science, Data Science, Systems Engineering, Public Policy, Public Health, or other relevant social science field with at least four-five years of demonstrated experience in relevant fields, as applied to developing countries
- Academic knowledge of and direct field experience with health, nutrition, and other social sector programming in L&MICs
- Research experience and in-depth knowledge of evaluation methods applied in causal attribution
- Experience with using alternative data sources (such as big data or administrative data) in evaluations
- Demonstrated expertise in using non-traditional evaluation methodology as described in the Responsibilities section
- Experience conducting impact evaluations for policy and institutional reform projects strongly preferred
- Previous experience conducting a systematic scoping review preferred
- Excellent teamwork skills as well as ability to take initiative and work independently as needed
- Excellent oral and written communication as well as presentation skills in English; with professional proficiency in additional languages highly preferred
- Excellent organizational skills, attention to detail, and the flexibility and willingness to adapt to shifting priorities and deadlines
We will recruit one or more individuals to perform the identified tasks. 3ie is an equal-opportunity employer committed to equality and diversity. We do not discriminate based on sex, age, religion, ethnicity, caste, sexual orientation or for being differently abled. We particularly encourage ethnic minorities and differently-abled persons to apply.
Terms of employment
Candidate should be available to start work as soon as possible. The contract and daily rate will be discussed between final candidates and 3ie. Contracts will be for a maximum of one year and up to 20 days.
Our policies and procedures reflect our commitment to safeguarding children and vulnerable adults from abuse. We follow a zero-tolerance policy for any form of bullying or harassment in the workplace.
How to apply
Please apply by e-mail to firstname.lastname@example.org and include ‘3ie Complex Evaluation Specialist Consultant’ in the subject line. The applicant must provide, at a minimum*, the following information:
- A cover letter, not exceeding one page, highlighting your qualifications and experience relevant to the terms of reference
- Curriculum vitae (not to exceed four pages)
- Writing sample (preferably a sample of a proposal, research paper, or other technical document that you alone authored)
- Contact information for three professional references
*Incomplete applications will not be considered.
Only shortlisted candidates will be contacted. Applications will be reviewed on a rolling basis and the position will remain open until filled.