Series Evaluation Insights for Policy and Programming
Evaluation Synthesis: STDF/PG/350 — Global Phytosanitary Manuals, Standard Operating Procedures and Training Kits Project
Context
Phytosanitary capacity is foundational to agricultural trade. Countries that lack documented regulatory procedures, trained inspection staff, and functioning plant health systems face higher trade costs, increased consignment rejections, and reduced market access. This project addressed a critical institutional gap: the absence of globally endorsed technical guidance materials to help National Plant Protection Organizations (NPPOs) in developing countries implement the International Standards for Phytosanitary Measures (ISPMs) issued by the International Plant Protection Convention (IPPC).
The IPPC is the international standard-setting body recognized by the WTO under the Agreement on the Application of Sanitary and Phytosanitary Measures (SPS Agreement). Its standards govern how countries protect their agricultural sectors from pests and diseases introduced through international trade. Yet for decades, these standards existed without practical, country-accessible guidance on how to implement them. Phytosanitary Capacity Evaluations (PCEs) conducted in over 70 developing countries prior to 2012 consistently identified the lack of documented procedures as the principal weakness in national plant health systems.
The project therefore had clear and urgent strategic relevance at the global level. It was submitted by Côte d’Ivoire, Jamaica, Malaysia, and Sudan, and implemented by the IPPC Secretariat, hosted at the Food and Agriculture Organization of the United Nations (FAO) in Rome. The total project value was USD 822,000, of which USD 672,000 was provided by the Standards and Trade Development Facility (STDF), with the remaining resources contributed by the IPPC Secretariat as in-kind support.
Intervention Overview
The project ran from February 2012 to July 2015, with two no-cost extensions granted by the STDF Working Group totaling 18 additional months. Its immediate objective was to enhance the capacity of developing country NPPOs to implement the IPPC and ISPMs by providing internationally accepted technical resources, including manuals, operational procedures, and training kits.
The intervention was structured in three phases. Phase I involved a global call for existing resources, through which stakeholders contributed over 300 materials comprising 54% diagnostic protocols, 27% pest information, 8% manuals and guides, and 11% other resources. Phase II covered needs assessment and gap analysis by the IPPC Secretariat and the Capacity Development Committee (CDC). Phase III focused on the development and testing of priority resources for NPPOs identified during Phase II.
A distinctive feature was the collaborative, volunteer-based production model. International phytosanitary experts donated their time to draft and peer-review the materials, while their employing institutions provided logistical support. The project also established the “Phytosanitary Resources” website (www.phytosanitary.info) as an open-access repository. The CDC, composed of technical representatives from the seven FAO regions, served the dual role of project steering committee and technical review body.
By the end of the project, the IPPC had produced 14 Guides, 7 Training Kits, and 5 Fact Sheets covering core NPPO functions: import verification, export certification, plant pest surveillance, diagnostics, pest risk analysis, NPPO establishment and operation, and stakeholder management. These were the first globally endorsed IPPC capacity development materials ever produced.
Evidence: What Worked
Output Delivery Beyond Targets
The project substantially overachieved on its output indicators. A minimum of 20 documents were to be available on the IPPC portal by the end of Year 2; this target was met in 2014. A total of 14 Guides, 7 Training Kits, and 5 Fact Sheets were developed and published, a significantly larger body of materials than originally planned. The IPPC Secretariat and CDC members committed volunteer time far beyond contractual requirements, and paid and unpaid contributors alike delivered beyond what was budgeted.
The manuals were technically sound, fully compatible with the WTO SPS Agreement and the IPPC Convention, and passed the IPPC’s own quality review process. They became the first globally endorsed set of IPPC guidance tools, filling an institutional gap that had existed since the ISPMs were first adopted in 1951. This is a permanent and durable contribution to global phytosanitary governance.
Knowledge Use in Select Countries and Regions
Despite limited outreach, documented use cases confirm that the materials have had measurable operational value in several contexts. In Egypt, IPPC experts used the Market Access Manual in a 2015 workshop on phytosanitary measures in grain imports, helping to resolve a major trade dispute involving wheat consignments from Russia, the United States, Ukraine, and France. In Libya, a USD 3 million FAO-supported phytosanitary project drew extensively on the manuals to establish an NPPO from scratch. In Suriname, the manuals supported the preparation of national policy papers required by a major lending institution to qualify for agriculture sector funding.
The Russian translation of several guides, financed through another STDF project (STDF/PG/316 in Azerbaijan), enabled uptake in the Caucasus and Central Asia region. Georgia used the manuals to prepare a national Phytosanitary Capacity Development Strategy. Uzbekistan used the Russian-language manuals as a reference base for national adaptation. Azerbaijan reported progress in phytosanitary legislation following the FAO regional technical assistance project.
An NPPO survey conducted during the evaluation found that 18 of 26 responding NPPOs from developing countries confirmed awareness and use of the manuals. The Export Certification Guide and the Import Verification Guide were the most widely used, with the Plant Pest Surveillance Guide also rated highly.
Cost Efficiency of Production
The production model was economically efficient. The average cost per manual was approximately USD 50,000 for editing and USD 8,000 for publishing — substantially lower than comparable FAO field manuals produced through standard institutional procurement. The expert volunteer system, while posing scheduling risks, delivered high-quality technical content at minimal direct cost to the project.
Evidence: What Did Not Work
Outcome and Impact Not Achieved
The project did not achieve its outcome and impact objectives. None of the three outcome indicators — reduced consignment rejections, improved pest outbreak reporting, and increased positive implementation reports — were monitored or reported at any stage of the project. The Terminal Report makes no mention of outcome indicators at all, a significant oversight in results-based management. The contribution to STDF’s higher-level objectives (market access, domestic SPS improvement, poverty reduction) remains hypothetical and cannot be causally attributed to the project.
The core problem was a broken theory of change. The project assumed that publishing globally endorsed materials would lead to their adoption and use by NPPOs in developing countries. However, this assumption disregarded the institutional reality of NPPOs: chronic lack of training budgets, frequent staff turnover, limited language access, and an absence of implementation support mechanisms. Publishing a manual does not translate into its use without structured dissemination, technical assistance, and follow-up.
Inadequate Outreach and Dissemination
The planned Activity 5 (promotion and use of the technical resources) was never adequately funded or implemented. The IPPC submitted a follow-up proposal (STDF/PG/402) for training-of-trainers on the manuals, but it was rejected by the STDF Working Group. No alternative mechanism for systematic outreach to beneficiary NPPOs was established. Manuals were launched at Commission on Phytosanitary Measures (CPM) meetings and regional events, but were not actively disseminated to the RPPOs and NPPOs.
The Phytosanitary Resources Website attracted over 42,000 users between January 2014 and April 2018, with usage peaks coinciding with the “Pests without Borders” photo competition rather than manual downloads. Most traffic originated from developed countries, with Rome-based users (the IPPC Secretariat itself) accounting for the highest volume. The website was taken offline on 14 May 2018 due to FAO IT security concerns, leaving over 300 contributed resources inaccessible.
Translation Gap
Translation beyond English was achieved only for Russian and only because a separate STDF project provided financing. Despite the IPPC Secretariat committing in the project document to mobilize funds for translation into other official UN languages, no progress was made for Arabic, French, Spanish, or Chinese. Many survey respondents from Francophone African countries specifically noted language as a barrier to use.
Governance and Management Weaknesses
The CDC played a dual role as both steering committee and technical review body, which created structural inefficiencies. As a steering committee, it lacked project management expertise; as a technical body, its enthusiasm led to scope expansion that the available budget could not support. Staff turnover at the IPPC Secretariat disrupted implementation: the initial project manager resigned mid-project, and the replacement manager faced challenges re-engaging service providers with no knowledge transfer.
Lessons Learned
Several practical insights emerge from this evaluation with broad applicability to knowledge-based capacity development projects.
The “global manual” model has a ceiling. Producing endorsed global standards guidance is necessary but not sufficient for improving national SPS systems. The project addressed a global supply problem without adequately addressing the national demand side. NPPOs did not participate substantively in defining priorities; the four requesting countries provided political support but not implementation partnership. Future projects that produce knowledge assets must embed dissemination, technical assistance, and monitoring from the design phase.
Community of practice is a strategic asset. The phytosanitary expert community mobilized voluntarily and contributed significant intellectual labor to the project. This social capital (built through the IPPC’s convening power) is genuinely valuable and should be actively maintained. Once the website went offline and the project closed, the community of practice lost its shared platform. Preserving and energizing this network is as important as maintaining the documents themselves.
Website sustainability requires dedicated resources. A knowledge-sharing platform cannot sustain itself through project funding alone. The offline transition of phytosanitary.info illustrates how institutional fragility, FAO IT security constraints, and post-project resource gaps combine to undermine digital knowledge infrastructure. Future projects must budget explicitly for the ongoing technical and editorial maintenance of any platform created.
Staff turnover is a systemic risk at both the IPPC and in NPPOs. High staff turnover in the IPPC Secretariat and in developing country NPPOs created dual vulnerabilities: the IPPC lost institutional memory for project implementation, while NPPOs lost personnel who had been trained on or introduced to the manuals. The availability of reference materials in a stable, accessible form is therefore even more important to mitigate the effects of turnover. Projects should build in mechanisms for knowledge continuity.
The absence of SMART outcome indicators undermined accountability. The logframe outcome indicators were not reformulated to be specific, measurable, achievable, relevant, and time-bound. Without baseline data and a monitoring plan, it was impossible to determine whether NPPOs had improved their capacity to manage national plant health systems. This is a foundational evaluation design failure that should be addressed in all future STDF projects.
Implications for Policy and Programming
For the IPPC Secretariat: The IPPC should institutionalize a strategy for the promotion and use of its guides and training materials, not just their production. This means identifying and formalizing partnerships with FAO field programmes, RPPOs, academic institutions, and South-South cooperation initiatives that can embed the manuals in technical assistance and training activities at national level. The IPPC should activate a “helpdesk” function open to NPPOs, the private sector, and other phytosanitary actors to respond to implementation queries. The participation of developing country representatives in the Implementation and Capacity Development Committee (IC) should be financed from core IPPC or FAO resources rather than depending on project-specific funding.
For the Implementation Committee (IC): The IC should establish a coherent strategy for the promotion, regular updating, and monitoring of existing manuals before embarking on additional material development. A comprehensive Procedural Manual that integrates and cross-references the current guides should be considered as a more sustainable alternative to a growing catalogue of standalone publications. Monitoring of manual use should be built into the IC’s regular reporting functions.
For the STDF: STDF should strengthen the requirement that projects include SMART outcome indicators, a results monitoring plan, and a dissemination strategy as non-negotiable elements of project design. Future SPS knowledge projects should include a distinct implementation and outreach component, with dedicated budget. STDF’s Prioritizing SPS Investments for Market Access (P-IMA) guides could be systematically linked with IPPC manuals to offer NPPOs an integrated decision-support toolkit. STDF should also ensure that its Virtual Library serves as a stable, searchable clearinghouse for all SPS capacity development resources.
For the broader donor community: Donors should recognize that phytosanitary infrastructure deficits in developing countries are chronic and structural, not episodic. Investments in SPS capacity need to be sustained, multi-year, and coordinated across bilateral and multilateral channels. South-South and triangular cooperation should be actively promoted as cost-effective modalities for sharing phytosanitary expertise between developed-country NPPOs and their developing-country counterparts. Donors should invest in multilateral solutions and avoid fragmented, project-by-project approaches.
Potential for Scaling and Transferability
The IPPC guides and training kits themselves have high scaling potential: they are technically endorsed, globally applicable, freely accessible online, and already in use in multiple regions. The primary bottleneck is not the quality of the content but the absence of a systematic delivery mechanism. Where a regional technical assistance infrastructure exists, as demonstrated in the Caucasus and Central Asia case, the manuals can effectively support national capacity building.
The postgraduate training course piloted three times at CIHEAM Bari in Italy offers a tested model for systematic, institution-based delivery of the IPPC guide content. This model could be adapted and replicated by other regional centers in Africa, Asia, or Latin America, provided adequate financing is secured. IPPC is actively seeking an academic partner for a full postgraduate programme on phytosanitary regulation.
For the volunteer-based production model, scalability depends on sustaining the community of practice. This model works where a convening organization commands global technical authority, where experts can be mobilized through professional networks, and where a neutral platform facilitates exchange. These enabling conditions exist within the IPPC system but require active stewardship. A formal knowledge-sharing mechanism, technically sound and financially stable, would allow the IPPC to mobilize expert contributions beyond project cycles.
Methodological Notes
This synthesis is based on an ex-post evaluation conducted by an independent evaluator between July and December 2018. The evaluation applied a Theory of Change framework combined with mixed methods: document review, key informant interviews conducted during a mission to FAO Rome (15 to 18 October 2018), teleconferences with key informants, an NPPO survey via Google Forms distributed to 146 countries, and website analytics from Google Analytics for the period January 2014 to April 2018.
The evaluation was grounded in the OECD DAC principles of impartiality, independence, credibility, usefulness, and stakeholder participation, and adhered to the standards of the German Evaluation Society (DeGEval). The evaluator confirmed no prior involvement in the project.
Key methodological limitations include a low NPPO survey response rate (25 responses from 146 countries contacted), driven by staff turnover and low project visibility at national level. Many addresses provided by IPPC were outdated. The Phytosanitary Resources website was offline at the time of evaluation, preventing direct access to usage content; only aggregate Google Analytics data were available. Attribution of observed phytosanitary capacity improvements to the project specifically is constrained by the absence of a monitoring baseline and the concurrent influence of other technical assistance programmes.
Stakeholder Perspectives
IPPC Secretariat staff reported that the project overachieved on outputs due to exceptional commitment by all participants, paid and unpaid. Several CDC members noted that, while technically rewarding, the dual role of steering committee and technical reviewer created governance confusion and slowed decision-making.
NPPO focal points in developing countries who responded to the survey confirmed that the Export Certification Guide and Import Verification Guide were the most practically useful, while 18 of 26 respondents indicated awareness and use. The most frequently reported barriers to use were lack of training budget (cited by 18 of 23 respondents), lack of available guidance on how to implement the manuals, and insufficient time for training.
One Contact Point offered a particularly instructive observation: if NPPOs had been involved in producing the manuals themselves (contributing their best practices and receiving support to consolidate them) ownership and usage would have been substantially higher. This aligns with the broader finding that demand-driven, participatory approaches outperform supply-driven material production for capacity development at national level.
Further Resources and Links
Implementing Organization
International Plant Protection Convention (IPPC) Secretariat — https://www.ippc.int
IPPC Guides and Training Materials
https://www.ippc.int/en/core-activities/capacity-development/guides-and-training-materials/
Funding and Coordination
Standards and Trade Development Facility (STDF) — https://www.standardsfacility.org[^2]
Project Page
STDF/PG/350 project documentation — https://www.standardsfacility.org/PG-350
Implementing Host Organization
Food and Agriculture Organization of the United Nations (FAO) — https://www.fao.org
Related Capacity Development Tool
IPPC Phytosanitary Capacity Evaluation (PCE) Tool — https://www.ippc.int/en/core-activities/capacity-development/phytosanitary-capacity-evaluation/
Related Knowledge Platform
STDF Virtual Library — https://www.standardsfacility.org/virtual-library
Regional Training Model
CIHEAM Bari, Italy — https://www.iamb.it
IPPC Strategic Framework 2020-2030
https://www.ippc.int/en/core-activities/governance/ippc-strategic-framework/[^3]
Report Citation
Weitzenegger, Karsten. Ex-post Evaluation of the STDF Project „Global Phytosanitary Manuals. Standard Operating Procedures and Training Kits Project“ (STDF/PG/350), STDF Standards and Trade Development Facility, WTO Geneva 2019. standardsfacility.org/sites/default/files/STDF_PG_350_Evaluation_Report_Jan19.pdf. Accessed 20 May 2026.
Disclaimer: The author participated in this evaluation. The opinions expressed are solely those of the author and cannot be attributed to any affiliated organizations. Portions of the text and images were supported by artificial intelligence.