3ie provides answers to frequently asked questions about Systematic Review calls
- Do I need to submit an 'Expression of Interest' (EoI) or a 'Letter of Interest' (LoI) or can I submit a full proposal?
- What are the steps for submitting an application?
- Can I work on my application online, save it and return to it later?
- The application form allows for ‘Title of the Review (if different from Review Question)’. What does this mean?
- Can one organisation be awarded more than one grant?
- Can more than one affiliate of an organisation submit proposals?
- Does 3ie welcome bids from consortia of researchers?
- Are ‘match-making’ services available for teams seeking additional members?
- Do I need to submit a separate application for each question on which I want to bid?
- What are the focus countries under this call for proposals?
- What is the primary audience of the systematic reviews?
- What is the deadline for submission of proposals?
- When will successful applicants hear back?
- How much does a systematic review cost? Are there any grant limits?
- What resources does a systematic review entail?
- How do I know if my institution qualifies for this Request for Proposals?
- Does 3ie promote certain systematic review methods and approaches?
- Does 3ie only want systematic reviews of randomised-control trials (RCTs)?
- What kind of methodological standards are expected from systematic reviews?
- On which criteria will the quality of proposals be assessed?
- What support can 3ie give for preparing proposals?
- What are the acceptable indirect costs?
- What are the acceptable direct costs?
- Where can I find out more about systematic reviews?
- Where can I find out more about the stakeholder engagement and communication plan (SECP)?
- What do I do if my question is NOT answered here?
Frequently asked questions
Do I need to submit an 'expression of interest' (EoI) or a 'letter of interest' (LoI) or can I submit a full proposal?
You do not need to submit an EoI or LoI. You go directly to the full proposal online and complete it.Back to top
What are the steps for submitting an application?Back to top
Can I work on my application online, save it and return to it later?
Yes, the 3ie online application allows you to work on your application without submitting it. On hitting the ‘Save’ button, a copy of your information is saved in the form itself. You will be able to return to it at another time and continue working on it, and save it as many times as you wish. It is also possible to print your application while you are working on it or after it has been submitted. Unless you hit the submit button, your application will not be considered complete and submitted for review.Back to top
The application form allows for ‘Title of the study if more focused or broader than the review question selected above’. What does this mean?
Some of the proposed systematic review questions might be usefully broken down into smaller questions. 3ie advises you to propose appropriately focused review questions for questions on broad topics where substantial evidence exists. If you think you need to reword the proposed question or suggest a sub-question, please put this on the form. The winning bidders will be expected to agree on the scope and focus with 3ie, funders of this call and an advisory group at the review scoping stage.Back to top
Can one organisation be awarded more than one grant?
YesBack to top
Can more than one affiliate of an organisation submit proposals?
YesBack to top
Does 3ie welcome bids from consortia of researchers?
3ie encourages applicants to collaborate with partners from other organisations to complete their teams, for example teams comprising systematic review specialists and subject area specialists. 3ie is only able to sign grants with single organisations, and if grantees wish to have sub-agreements with other teams, this needs to be clearly budgeted as such.Back to top
Are ‘match-making’ services available for teams seeking additional members?
Teams wishing to recruit additional members or researchers who can provide relevant information or skills to review teams are encouraged to sign up to the SRs-in-international-development email listserv to post information and requests: email@example.comBack to top
Do I need to submit a separate application for each question on which I want to bid?
YesBack to top
What are the focus countries under this call for proposals?
3ie systematic reviews focus on evidence from, and applicable to, low- and middle-income countries. For some proposals, the target region may be further specified geographically or according to low-income country, fragile states or least-developed country classifications. Where relevant, evidence and studies from non-developing country settings may be incorporated into the review if there is a strong rationale for doing so.Back to top
What is the primary audience of the systematic reviews?
International policymakers, advisors, implementation programme managers and evaluators are key audiences for the reviews.Back to top
What is the deadline for submission of proposals for systematic review call 8?
The deadline for submissions is: 22 May 2015 12 noon GMT, 07:00 EST (GMT-5), 16:30 IST (GMT+4½)Back to top
When will successful applicants hear the results?
Successful applicants will be contacted in June 2015. Unsuccessful applicants will not be notified. But, by the end of July 2015, they will be able to access their peer review forms, without the scores, by logging onto the 3ie online application system.Back to top
How much does a systematic review cost? Are there any grant limits?
As an approximate guide, the average budget of studies financed under the previous call for proposals was US$82,500. However, the cost of a systematic review will depend on the size of the evidence base to be reviewed and the methods of synthesis used. If there are many studies to review, and/or the study draws on a range of evidence (for example, including both counterfactual evidence and broader evidence to answer separate review sub-questions), the review might cost more. In previous calls, budgeting mixed method reviews has proved particularly challenging. 3ie urges careful attention to estimating budget costs as accurately as possible in all instances.Back to top
What resources does a systematic review entail?
There should be sufficient staff to ensure the independent double reading of full text articles, data extraction and quality appraisal of included studies, with third party referral in case of disagreement. Teams are required to consult search librarians or information specialists as part of the process of developing a protocol. In order to conduct a comprehensive search, teams will need full-text access to journals, an academic library and the internet, and access to reference management software. Access to statistical software packages for meta-analysis or software packages for qualitative data analysis are an advantage. Successful teams are also expected to form an advisory group.Back to top
How do I know if my institution qualifies for this Request for Proposals?
Any organisation that may legally accept a grant agreement can apply. Study teams are assessed based on qualifications and experience of team members, as indicated in the review criteria found in the Request for Proposals.Back to top
Does 3ie promote certain systematic review methods and approaches?
3ie partners with the Campbell Collaboration in the production of systematic reviews, and accordingly reviews of effectiveness must be conducted to Campbell standards. However, 3ie takes a broad perspective on the types of questions and evidence on which a systematic review can be based. Evidence of effectiveness requires mixed methodological approaches, including quantitative and qualitative studies, using experimental and quasi-experimental designs, in controlled and real life settings. In practice this means reviews: (1) are based on high-quality evidence appropriate to the research question; (2) set the outcome in the broader context of the underlying programme theory, reporting evidence on all links in the causal chain, not only final (endpoint) outcomes. Hence teams may wish to synthesise broader factual and counterfactual evidence, and quantitative and qualitative evidence, for example on barriers and facilitators of change; and (3) examine the variation and heterogeneity of reported outcomes, not only their central tendency (average effect size).Back to top
Does 3ie only want systematic reviews of randomised-control trials (RCTs)?
No, the type of study and method of synthesis must be appropriate to the question. Studies examining causal effects should use counterfactual methods. RCTs must be included where they exist, but quasi-experimental methods are usually admissible studies of intervention effects. In addition, simulation studies may be admissible for causal studies examining society- or economy-wide effects. For more information about quantitative evaluation design, see here and refer to Shadish, Cook and Campbell, 2002, Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Reviews should draw on broader factual evidence including that based on qualitative or mixed methods to understand mechanisms or factors explaining why something works or does not work in a particular context.Back to top
What kind of methodological standards are expected from systematic review?
There are many guidelines available for conducting systematic reviews:
- International Development Coordinating Group (IDCG) Guidelines page
- Journal of Development Effectiveness Special Issue on Systematic Review Methods
- Campbell Collaboration methods resources
- Cochrane Handbook; see also Cochrane Qualitative and Implementation Methods Group Supplemental Handbook Guidance
On which criteria will the quality of proposals be assessed?
The quality of proposals will be assessed according to whether they are substantively important and methodologically appropriate. All of the criteria and weights are found in Appendix B of the Request for Proposals.Back to top
What support can 3ie give for preparing proposals?
In order to ensure equal opportunity, 3ie staff are unable to provide guidance to specific teams or be named as part of teams submitting proposals. 3ie has put resources on this website for those wanting to submit proposals in response to this Request for Proposals.Back to top
What are the acceptable indirect costs?
Please read the 3ie Indirect cost guidelines.Back to top
What are the acceptable direct costs?
Please read the 3ie Direct cost guidelines.Back to top
Where can I find out more about systematic reviews?
The websites of the Campbell and Cochrane Collaborations contain guidelines on producing systematic reviews and libraries of reviews. These resources, and others, are listed on this site under useful links and resources.Back to top
Where can I find out more about the stakeholder engagement and communication plan (SECP)?
3ie is in the process of revising and updating stakeholder engagement guidance. A draft version has been posted on the How to apply page. Submission and approval of the SECP is a prerequisite for successful applicants. The final SECP template and guidance will be made available to awardees over email.Back to top
What do I do if my question is NOT answered here?
Questions that are unanswered can be sent to firstname.lastname@example.org by 28 April 2015. Answers will be posted on the call page within 3 working days.