Series Evaluation Insights for Policy and Programming
Meta-analysis of evaluations on combating poverty and SDGs in Costa Rica
Context
The meta-analysis was conducted in Costa Rica in 2018 at the request of FOCEVAL and the SDG Technical Secretariat at MIDEPLAN, in a context where the country sought to align its National Evaluation Agenda with the implementation of the 2030 Agenda. The study is situated within the priority area of combating poverty, which mainly encompasses SDGs 1, 2, 3, 4, 5, 8, 10, 11, and 16. The development of the National Pact for the SDGs, signed in 2016, provided a policy framework that emphasizes the fight against poverty, inequality, and the exclusion of vulnerable groups such as women, indigenous peoples, Afro-descendants, LGBTI people, and people with disabilities.
The document is based on 25 recent evaluations of public policies, programs, and projects in Costa Rica relevant to poverty reduction, many of which were framed within the 2015-2018 National Evaluation Agenda, and others funded by sources such as FODESAF, the MDG Fund, and United Nations organizations. These evaluations cover sectors such as social protection, employment, education, health, rural development, gender equality, and housing, providing a relatively broad overview of Costa Rican social policy. At the same time, the analysis acknowledges that the set of evaluations does not encompass all relevant interventions in the country, and therefore its findings are more indicative than exhaustive.
Intervention Overview
The meta-analysis aims to systematize achievements, favorable and unfavorable factors, and recommendations related to combating poverty, as well as to identify the contributions of the evaluated interventions to the Sustainable Development Goals (SDGs) prioritized by Costa Rica. Its specific objectives include assessing results in terms of relevance, effectiveness, impact, and sustainability; identifying blind spots in SDG information and coverage; and defining priority areas for future evaluations of the National Assessment Agenda. The study also seeks to lay the groundwork for more in-depth analyses of priority areas of the 2030 Agenda, such as sustainable production and consumption, and sustainable infrastructure and communities.
The evaluative intervention is characterized by a data mining approach applied to evaluation documents, allowing for the systematic analysis of keywords and codes associated with poverty dimensions and SDGs. The period of the underlying evaluations spans approximately 2011 to 2018, with a focus on the implementation phase of the National Evaluation Strategy (ANE) from 2015 to 2018. The meta-analysis examines evaluations of flagship programs such as the Bridge to Development strategy, the Avancemos program, the National Childcare and Development Network, employment programs, gender equality policies, and housing and social protection programs.
Evidence : What Worked
The meta-analysis shows that assessments in Costa Rica cover a wide range of poverty dimensions, including well-being, education, employment, income, participation, health, rural conditions, and gender, indicating a relatively multidimensional understanding of the phenomenon. Keywords such as women, change, and need appear in 100% of the cases, while terms like well-being, education, income, and participation are recorded in more than 90% of the assessments, suggesting that these themes have been integrated across all areas. Furthermore, there is a notable presence of concepts related to equality, diversity, and partnerships, consistent with the 2030 Agenda’s focus on inclusion and partnerships.
Evaluations linked to the Bridge to Development Strategy stand out for their high thematic similarity with other interventions, indicating a greater degree of systematization in the approach to extreme poverty. Likewise, programs such as REDCUDI, Empléate, and the Business Organizational Development Incubator exhibit high levels of similarity, suggesting that they address similar dimensions of poverty, social protection, and employment from diverse perspectives. The co-occurrence analysis shows that the poverty code is closely associated with well-being, employment, need, women, change, health, rurality, and income, reflecting an evaluative narrative that recognizes multiple determinants of well-being.
Evidence: What Did Not Work
The study concludes that, despite the breadth of topics covered, the evaluation of poverty reduction efforts in Costa Rica is generally unsystematic, with little consideration given to the interactions between dimensions and sustainability goals. Each evaluation report tends to use its own set of criteria, without a robust common framework, which limits comparability between interventions and hinders the construction of an aggregate view of progress toward the SDGs. The lack of harmonization of criteria and methodological approaches results in a fragmentation of evidence that reduces its usefulness for national strategic planning.
The meta-analysis also identifies significant gaps in information and data quality, including heterogeneous formats, poorly harmonized administrative records, and large volumes of data lacking the necessary systematization to inform indicators and outcomes analysis. Certain codes, such as healthy living, basic products, reproductive health, older persons, empowerment, decent work, disadvantage, and governance, show low coverage in the evaluations, suggesting potential areas of insufficient attention in interventions or at least in the documentation of outcomes. There is also a lack of sex-disaggregated evaluations and a shortage of specific questions addressing the differentiated perspectives of women and men, despite the centrality of a gender approach in the SDGs.
Lessons Learned
A key lesson is that Costa Rica has made progress in incorporating multiple dimensions of poverty into its assessments, but it still lacks a truly comprehensive approach that captures the interrelationships among the SDGs and between social, economic, and environmental dimensions. The concentration of concepts such as women, change, and need in all assessments indicates sensitivity to gender issues and social dynamics, but the lack of systematic analysis of interactions limits the ability to identify synergies and trade-offs. Offs between policies. This suggests the need for more complex evaluative frameworks that go beyond the sum of isolated criteria.
Another lesson concerns the importance of standardizing evaluation criteria and questions to increase comparability and aggregate results, especially in the context of monitoring the 2030 Agenda. The meta-analysis shows that without a minimum set of shared criteria and a common structure for documenting results, the potential contribution of evaluations to decision-making is diminished. At the same time, the exercise reveals that innovative approaches such as text analysis and data mining can add value by leveraging large volumes of existing evaluation information.
Implications for Policy and Programming
For public policy, the findings reinforce the need to align the National Evaluation Agenda with the 2019-2022 National Development and Public Investment Plan and with the country’s strategic vision for 2050, so that evaluation generates evidence directly usable for planning and budgeting decisions. This implies strategically selecting interventions to be evaluated, prioritizing those that are innovative, critical for poverty reduction, and highly visible to citizens, as well as strengthening the connection between evaluation and accountability processes. It also raises the advisability of shifting from evaluations focused on individual projects to thematic or instrument-based evaluations capable of capturing aggregate effects in relation to specific SDG targets.
From a programmatic perspective, it is recommended that key evaluation questions be formulated that explicitly link to the principles of the 2030 Agenda, including universality, comprehensiveness, the interconnectedness of goals, and the principle of leaving no one behind. Future evaluations should systematically incorporate criteria of relevance and impact with respect to the 17 SDGs and explicitly analyze how different policies may support or undermine the objectives of poverty reduction, inequality reduction, and sustainability. It is also suggested that the usefulness of evaluations for policymakers and citizens be increased through more active dissemination processes and adapted communication formats, leveraging existing SDG information platforms.
Potential for Scaling and Transferability
The meta-analysis approach to evaluations using text mining has significant potential for replication and adaptation in other countries with extensive corpora of evaluation reports but lacking an integrated view of their contribution to the SDGs. The use of tools such as QDA Miner , WordStat , and CATMA facilitates pattern extraction, the identification of thematic clusters, and the detection of information gaps without necessarily requiring new data collection. This type of exercise can be particularly useful in contexts where analytical capacity is limited and where reorganizing existing evidence is more feasible than funding additional studies.
However, the transferability of this approach depends on certain enabling conditions, such as the availability and accessibility of evaluation reports in digital format, the existence of a national evaluation coordinating entity, and the political will to use the findings in policy planning and prioritization. In the case of Costa Rica, the role of the SDG Technical Secretariat within MIDEPLAN and capacity-building projects like FOCEVAL demonstrate that meta-analysis is most effective when an institutional infrastructure exists to monitor it and translate results into decisions. Other interested countries should also consider the necessary investment in technical capacity for using qualitative and text analysis software.
Methodological Notes
The meta-analysis uses a data mining approach applied to 25 evaluations selected for their relevance to the priority objective of combating poverty and the associated Sustainable Development Goals (SDGs), encompassing interventions from various sectors and funding sources. The process includes dataset selection, analysis of text properties, data transformation and preprocessing, application of data mining techniques and knowledge extraction, followed by the interpretation and validation of the resulting models. The selection of codes is based on a list of keywords linked to the SDGs, developed by MIDEPLAN, which includes both general terms on sustainability and words associated with specific goals.
Among the main tools used are QDA Miner for qualitative analysis, WordStat for text mining, CATMA for digital annotation, and word cloud generators, which allow for the visualization of frequencies and co-occurrences. The study acknowledges several limitations, such as the heterogeneity in the length and structure of the reports, the dependence on each author’s linguistic style, and the inability to capture information contained in non-textual visualizations. Furthermore, the analysis is limited to a corpus of 25 cases and does not aim to exhaustively represent all policies relevant to the SDGs in Costa Rica, but rather to offer initial input to guide future evaluations.
Stakeholder Perspectives
The SDG Technical Secretariat at MIDEPLAN emerges as a central actor in the demand for the meta-analysis, reflecting an institutional interest in using evaluation as a tool for monitoring national goals and for aligning public policies with the 2030 Agenda. FOCEVAL, funded by the BMZ and jointly implemented by DEval and MIDEPLAN, contributes the perspective of international cooperation aimed at strengthening evaluation capacities and consolidating a culture of evidence-based decision-making. The involvement of universities, research centers, and national and international consulting firms in the 25 underlying evaluations demonstrates a diverse evaluation community, but also reinforces the heterogeneity of approaches and criteria documented by the meta-analysis.
The report underscores the need for more active involvement of non-state actors, including civil society organizations, faith-based organizations, the private sector, and international agencies, in the evaluation of policies linked to the 2030 Agenda. It also highlights that ensuring no one is left behind requires strengthening the voices of marginalized groups such as people with disabilities, youth, women, and people living in poverty, by combining quantitative data with qualitative methods that capture the stories behind the indicators. In this sense, evaluation is conceived not only as a technical tool but also as a political process that can empower vulnerable actors and open spaces for dialogue among sectors.
Key Data Points and Indicators
In the corpus of 25 evaluations, 43 keywords associated with combating poverty and the prioritized SDGs were identified, with a total of more than 17,000 relevant words repeated more than once. The code coverage shows that terms such as women, change, and need appear in 100% of the cases, while well-being, education, income, and participation are observed in at least 96 % of the evaluations. In contrast, codes such as resilience do not appear in any case, and others such as healthy life, basic goods, reproductive, and older adult show coverage of less than 15%.
Case similarity analysis suggests that evaluations of the Bridge to Development strategy show the broadest relationship with other reports, while interventions such as EDU INCL and Banvhi appear relatively isolated in terms of keyword patterns. Furthermore, common keywords such as program, evaluation, people, development, care, process, information, work, population, and outcomes are frequently identified throughout the corpus, reflecting the institutional and operational nature of the documents. The dataset includes reports of varying lengths, from approximately 26 to 567 pages, which affects both the information density and the number of words analyzed.
Further Resources and Links
The report identifies numerous complementary resources that can support further analysis and international comparison, including documents from the Costa Rican SDG Technical Secretariat on the country’s situation, public policy inventory, and multi-level SDG governance. It also refers to global reports and methodological guides, such as evaluations of United Nations cooperation frameworks, documents from EvalSDGs , IIED, and the Global Partnership. for Sustainable Development Data, as well as a meta-analysis of sustainability in German development cooperation prepared by DEval. At the national level, portals such as ODS Costa Rica, FOCEVAL, and the MIDEPLAN website stand out, as they compile documentation on policies, evaluations, and progress related to the SDGs.
For those wishing to replicate or adapt the approach, the report provides references to software tools used in the analysis, such as QDA Miner and WordStat , as well as international platforms such as Sustainable Development Knowledge United Nations Platform . The annexes also include detailed lists of the 25 base cases, secondary documents consulted, and specific results of frequencies and co-occurrences, which can be reused for further analysis.
Report Citation
Weitzenegger, Karsten. Meta-Análisis de Evaluaciones en Vista de los Objetivos de Desarrollo Sostenible (ODS). Informe Final, DEval FOCEVAL Mideplan, San Jose 2018.
Disclaimer: The author participated in this evaluation. The opinions expressed are solely those of the author and cannot be attributed to any affiliated organizations. Portions of the text and images were supported by artificial intelligence.