Reset

Country: --   United States
City, State: Washington, DC

JOB DESCRIPTION

REQUEST FOR BIDS

**All candidates should submit their bids directly to korea_consultants@ndi.org and not complete the application online.**

The National Democratic Institute (NDI or the Institute) seeks to contract the services of a consultant(s)/firm to conduct an ex-post evaluation of NDI’s multi-year programming focused on North Korea. The consultant/s or consultant firm will work closely with NDI’s Asia-Pacific team to: 1) develop a full inception report that includes evaluation questions, proposed data collection and analysis methodology and protocols, instrumentation, sampling strategy, and data security and confidentiality protocols; 2) lead the evaluation data collection and analysis; and 3) produce a comprehensive report with recommendations for improving future program approaches based on lessons learned and program impacts thus far.

 

BACKGROUND

The rule of Kim Jong Un in the Democratic People’s Republic of Korea (DPRK or North Korea) carries forward a regime of repression established through generations of dictatorial rule. For more than half a century, conditioned and ill-informed by government propaganda and cut-off from the outside world, the North Korean public lacks basic knowledge about both human rights and the functioning of a democratic society. Moreover, the North Korean regime’s persistent practice of impunity and lack of accountability for egregious human rights abuses in the country continues to be an issue of grave concern. 

Through the significant efforts of the Seoul-based community of organizations and activists advocating for human rights and accountability of the North Korean government over human rights abuses to the international community, the UN Commission of Inquiry (COI) in 2014 issued a critical report that concluded the regime committed systematic human rights abuse violations amounting to crimes against humanity. Their efforts included documenting and analyzing human rights violations to prepare for transitional justice and disseminating this information to governments, international actors and the public to increase awareness about the need for accountability. With limited direct access to North Koreans, these organizations and activists face challenges in assessing the effectiveness of their programming. 

Since 2011, NDI has supported Seoul-based civil society organizations and human rights activists disseminating independent and pluralistic information into North Korea, and advocating for greater accountability of the North Korean government. NDI’s programs include promoting concerted efforts of groups to develop diverse and tailored content and innovative dissemination plans. NDI also provides human rights organizations a platform and technical assistance to expand strategic advocacy efforts by engaging with a broader network of local and regional partners and enhancing their ability to collect and document data on human rights abuses more effectively. NDI also provides technical assistance to its partner organizations on communication and public outreach skills and supporting staff development.

 

PURPOSE OF THE EVALUATION

This evaluation will cover four NDI programs with the first one starting in 2012. The purpose of the ex-post evaluation is to assess the relevance of these programs to the needs and priorities of the beneficiaries and coherence with other interventions with similar objectives; examine the effectiveness of program activities and outcomes in achieving the intended results and any potential impacts on the beneficiaries beyond the life of the project; and bolster evidence-based learning. The assessment will identify cause-and-effect relationships, and map out the complexity of actors and factors that influence and/or inhibit sustainable democratic systems. The ex-post evaluation is meant to capture insights, results and lessons learned from the Institute’s past work funded by the donor, and use them to inform and expand NDI’s future programming.  

 

EVALUATION CRITERIA

The evaluation will use the following modified criteria from the Organization for Economic Cooperation and Development’s Development Assistance Committee (OECD/DAC) criteria   to design the evaluation: 

  • Effectiveness of the interventions in achieving outcomes and objectives;

  • Relevance of the donor’s interventions and critical assumptions to development problems and the implementation environment;  

  • Potential sustainability of the program results after cessation of program activities;

  • Potential impact of the program on the beneficiaries beyond the life of the project; and

  • The coherence of the program with other interventions.   

 

EVALUATION METHODOLOGY 

The evaluation will use a modified Outcome Harvesting methodology to collect, analyze, validate and synthesize the changes that have occurred. Outcome harvesting collects evidence on what has been achieved and then works backwards to determine whether and how the program contributed to the change. The methodological approach will answer key evaluation questions related to progress towards the main lines of efforts while identifying cause-and-effect relationships between program activities and results; determining the extent to which results are sustainable; and developing a more complete understanding of the opportunities and challenges including the factors and actors that influenced or inhibited intended and unintended changes. Outcome harvesting will be done through a combination of desk review, key informant interviews (KIIs) and focus groups discussions (FGDs) either virtually or in person, and may include other methods to be specified in the full inception report. Given the scope of the evaluation, there may be a need to utilize other data collection analysis methods to address any questions that were not answered through the outcome harvest.

*Key Evaluation Questions will be provided to shortlisted consultants or firms

EXPECTED DELIVERABLES OF SELECTED CONSULTANT

  1. A Detailed Inception Report and Methodology: NDI will provide the consultant with program design documents including stakeholder and front end analysis, quarterly and annual program reports, data and information gathered through monitoring, program evaluations or other assessments, and any other program documents for desk review. Following the desk review, the evaluation team will hold a workshop with the NDI South Korea team to explore the findings from the desk review, the outcomes and mechanisms identified, as well as causal relationships and any alternate pathways of change. Given the scope of the evaluation, there may be a need to utilize other data collection and analysis methods to address key evaluation questions that cannot be answered through the outcome harvest. Based on these reviews and discussions with NDI staff, the consultant will develop an inception report that will include key evaluation questions, sub-questions, data collection tools, analysis methods, primary data sources and sampling strategy. The consultant will adhere to NDI’s approach for maintaining data security and data confidentiality. The draft inception report will be reviewed and approved by the Institute’s regional program team. The selected consultant will incorporate feedback and finalize the evaluation matrix as well as the key components of the inception report. The approved inception report will serve as the blueprint for the field portion of the ex-post evaluation.

  2. Data Collection and Analysis: The consultant will hold an orientation with NDI’s field team to introduce the translators, interpreters, and enumerators in harmony with the evaluation methodological protocols. While NDI will provide materials, the consultant will ensure that the enumerators and translators transcribe and/or translate data in an accurate, timely and reliable manner. The consultant and the evaluation team will conduct key informant interviews and focus group discussions, and any other methods defined in the inception report.  The consultant will then conduct a full analysis of the data from a variety of sources, collected by the mixed methods approach to develop a comprehensive report on NDI’s programming.

  3. Comprehensive Evaluation Report:  After completing the data analysis, the consultant will brief NDI and the donor on findings and prepare a draft evaluation report. The draft report will be reviewed by NDI and the donor for comments and feedback. After responding to the comments and feedback, the consultant will submit a final report with the raw data and collection tools. The final evaluation report should include the following:

    1. Executive summary of findings and conclusions;

    2. Introduction with background to the evaluation;

    3. Evaluation questions, methodologies and approach, scope, constraints, and limitations;

    4. Description of the project objectives evaluated;

    5. Data collection and analysis based on criteria and findings;

    6. Key findings and conclusions; and

    7. Annexes, including: TORs, work plan, data collection instruments and protocols, raw data references, documentation, etc.

 

CONSULTANCY TERMS

  1. The exact term of consultancy will be determined upon signature of contract. The contracted firm/consultant is expected to begin by February 1, 2021 and will ensure to complete the entire evaluation process by June 30, 2021 based on the suggested timelines below:

    1. Document Review & Evaluation Methodology: Approximately 1.5 months

      1. Review source documents, conduct pre-field work meetings, complete desk review 

      2. Hold workshop with NDI field office team 

      3. Prepare and submit draft and final inception report, with evaluation matrix and data collection tools

      4. Arrange logistics and schedule field work

    2. Data Collection & Analysis: Approximately 1.5 months

      1. In-field data collection 

      2. Complete data collection and preliminary analysis

      3. Presentation with the donor and NDI on initial findings and recommendations 

    3. Report on Findings: Approximately 1.5 months

      1. Prepare and submit draft evaluation report 

      2. Respond to the donor comments on report and submit final report 

  1. Due to the ongoing COVID-19 pandemic, there will be no travel to South Korea for consultants based outside of the country. All local travel from evaluation team members based in South Korea and their associated costs should be factored into the contract rate. The consultant is responsible for the filing and payment of all withholding and other taxes to which he/she is subject. He/she is not entitled to any fringe benefits from NDI, including but not limited to sick leave, health or life insurance and pension plan.

  2. NDI will make payments to the consultant upon receipt of invoices for work completed and DC approval of the final report. 

  3. All raw data collected and reports prepared would be considered property of NDI. 

 

EVALUATION TEAM ROLES, RESPONSIBILITIES, & QUALIFICATIONS

** Evaluators or evaluation firms with existing field operations and/or capabilities in South Korea will be given preference.

  • Lead Evaluator (TBD): The lead evaluator is responsible for the designing the outcome harvest and implementing the evaluation, including: methodological approach, data collection instrumentation, analytical frameworks, guiding the preliminary analysis, data analysis and drafting key elements of the final report. This will include the development and implementation of selected data collection tools and methods for conducting the evaluation.  The lead evaluator or evaluation firm will also be responsible for identifying and hiring local and international evaluation team members in close coordination with NDI and the donor, preparing and submitting a budget along with the inception report and coordinating with the NDI DC and Seoul staff. The criteria for the selection of this position include: 

  1. Must demonstrate a high level of knowledge and expertise in evaluating closed society programmings;

  2. Knowledge, expertise and professional background in foreign assistance evaluation and in democracy and human rights issues;

  3. Experience with a variety of quantitative and qualitative evaluation methodologies, such as how best to establish criteria for judging “success” and impacts of foreign assistance programs, the ability to analyze, synthesize and draw larger conclusions and lessons learned from various sources of data and findings;

  4. A minimum of ten years’ experience leading an M&E teams and demonstrate proven ability to design, implement, and evaluate foreign assistance programs of democracy and human rights programs; 

  5. Fluency in English (Korean is a plus). 

  • Evaluation Team Member/s (TBD): The evaluation team member/s will assist the lead evaluator in the development and implementation of the methodological approach, data collection instrumentation, analytical frameworks, guiding the preliminary analysis, data analysis and drafting key elements of the final report. The teams should include locally hired members who have experience conducting evaluations in South Korea. Moreover, the team should also ensure to contract its own translator to translate and back translate any materials necessary for the collection and analysis of data, including surveys, focus group protocols, PowerPoint slides, graphs and charts, as well as the qualitative responses from group discussions and/or surveys. The criteria for selection of these positions include:  

  1. Knowledge and expertise in evaluating closed society programming, and democracy and human rights issues;

  2. Deep knowledge of the country and culture, political processes, the civil society community;

  3. Must possess exceptional organizational and communication, and analytical and critical thinking skills, as well as resourcefulness and flexibility;

  4. Experience in evaluation design and implementation, including the development of instruments and methods;

  5. Written and verbal fluency in both English and Korean. 

  • Evaluation Support from NDI DC & South Korea Field Team (NDI/Asia): NDI’s DC and Seoul staff including the Washington DC-based M&E manager will support the evaluation team by providing desk research materials, program documents and data, identifying appropriate sources of data and counseling the evaluation team on the feasibility and appropriateness of their agenda given the program content, timing and context. In regards to processes and input from NDI staff, the DC and field teams will review and provide feedback into the inception report, evaluation matrix and data collection tools and complete methodology, and the draft evaluation report. Comments and concerns will need to be addressed by the lead evaluator within one month of submitting the inception report and the draft evaluation report.

PROPOSAL INFORMATION

The bidding process consists of two steps: 

  1. Consultants should submit bids that indicate the consultant and/or team’s interest and qualification to oversee and conduct the evaluation, which should include the following:

    1. CV of the proposed evaluation consultant or evaluation team;

    2. A cover letter outlining your interest and experience in the evaluation approaches;

    3. Example of past assessments; and

    4. At least two references. 

  2. NDI will then share the full scope of work with shortlisted candidates and request the submission of full proposals.

NDI will evaluate applicant’s documents based on the vendor’s experience and ability to meet the description of the product needed. Interested applicants should submit the initial documents outlined in Step 1 as soon as possible via email to korea_consultants@ndi.org. The submission deadline for full proposals is Monday, January 25, 2021.

ABOUT NDI

NDI is a nonpartisan, nongovernmental organization that responds to the worldwide quest for popular civic participation, open and competitive political systems, and representative and accountable government. Since its founding in 1983, NDI and its local partners have worked to establish and strengthen democratic institutions and practices by building political and civic organizations, safeguarding elections and promoting citizen participation, openness and accountability in government. You can learn more about the Institute at our website, http://www.ndi.org.

HOW TO APPLY

If you have any other questions, do not hesitate to contact us.

CONTACT US