Flaws and Errors Identified in the Institute for Research and Evaluation Report That Challenges Non-United States, School-Based Comprehensive Sexuality Education Evidence Base.

Comprehensive sexuality education (CSE) “aims to equip children and young people with knowledge, skills, attitudes and values that will empower them to: realize their health, well-being and dignity; develop respectful social and sexual relationships; consider how their choices affect their own well-being and that of others; and, understand and ensure the protection of their rights throughout their lives” [[1]UNESCO Office in HarareComprehensive Sexuality Education: Education for a Healthy Future. 8. UNESCO, 2018https://unesdoc.unesco.org/ark:/48223/pf0000376721Date accessed: September 28, 2022Google Scholar]. However, the implementation and scale-up of CSE programs are hampered by several challenges including deeply rooted opposition to CSE that has gained traction in recent years [[2]School-Based Sexuality EducationThe issues and challenges. Guttmacher Institute.https://www.guttmacher.org/journals/psrh/1998/07/school-based-sexuality-education-issues-and-challengesDate: 2005Date accessed: September 5, 2021Google Scholar]. Opponents of CSE claim that it is not effective in protecting young people and that it may cause harm. A recent review of CSE studies published by the Institute for Research and Evaluation (IRE)—“Re-Examining the Evidence for Comprehensive Sex Education in Schools” (referred to throughout this paper as “IRE report”)—is a recent manifestation of opposition campaigns waged against CSE [[3]Weed S.E. Ericksen I. Re-examining the evidence for comprehensive sex education in schools: A global research review.https://www.institute-research.com/CSEReport/Global_CSE_Report_12-17-19.pdfDate: 2019Date accessed: September 15, 2021Google Scholar]. The IRE report analyzed 43 non-United States school-based CSE studies included in three existing reviews. The authors of the report contend that the current evidence base does not support the effectiveness or acceptability of CSE, as evidenced by what they claim are high rates of program failure and harmful effects, and few cases of success. The IRE is actively disseminating this report to member states of the United Nation to influence policymaking and funding decisions. Their campaign has generated confusion and doubts about the established evidence on CSE. A recent webinar hosted by Family Watch International promoted the IRE report findings as grounds to oppose the renewal of the Eastern and Southern Africa Ministerial Commitment [[4]Family Watch InternationalDevious ESA commitment.https://familywatch.org/2021/12/01/devious-esa-commitment/#.Ya7CEL5KiM8Date: 2021Date accessed: December 5, 2021Google Scholar,[5]No author. CSE Research. Devious ESA Commitment. Published No date.https://deviousesacommitment.org/cse-research/Date accessed: December 6, 2021Google Scholar]. Their report has been shared through news outlets and used by other anti-CSE organizations as grounds to oppose the renewal of policies on CSE programs [4Family Watch InternationalDevious ESA commitment.https://familywatch.org/2021/12/01/devious-esa-commitment/#.Ya7CEL5KiM8Date: 2021Date accessed: December 5, 2021Google Scholar, 5No author. CSE Research. Devious ESA Commitment. Published No date.https://deviousesacommitment.org/cse-research/Date accessed: December 6, 2021Google Scholar, 6Johnston J. Is “comprehensive Sex education” effective - as its proponents claim? Focus on the family’s the daily citizen.https://dailycitizen.focusonthefamily.com/is-comprehensive-sex-education-effective-as-its-proponents-claim/Date: 2021Date accessed: October 27, 2021Google Scholar]. Because these findings undermine the case made for CSE and challenge the extensive body of research and evidence in support of its effectiveness, they warrant careful reanalysis. The authors of this commentary assessed the validity of the IRE report findings by reanalyzing the 43 included studies and investigating the extent to which the report’s methodology conforms to well-established scientific standards (this manuscript is in the process of being published in the Sexual and Reproductive Health Matters journal). This commentary provides a summary of key critiques of the IRE report. First, we found fundamental flaws and errors in the analytical framework underpinning the analysis conducted by the authors of the IRE report. The framework lacked crucial operational definitions of key indicators used to gauge program success, and there was substantial overlap across the indicator definitions. The framework arbitrarily privileged some indicators over others, assigning greater weight to studies that demonstrated program success on four indicators which they referred to as “key protective indicators.” This hierarchy was not explained or supported by evidence from prevention research or behavior change literature. In turn, this minimized intervention positive effects for studies that showed an impact on one or more of what the authors called “less protective indicators.” The framework also omitted important indicators such as those that gauge changes in knowledge and attitudes which can be precursors to behavior change and can be directly impacted by risk-based education programs [[7]Bettinghaus E.P. Health promotion and the knowledge-attitude-behavior continuum.Prev Med. 1986; 15: 475-491Crossref PubMed Scopus (219) Google Scholar,[8]Haberland N. Rogow D. Sexuality education: Emerging trends in evidence and practice.J Adolesc Health. 2015; 56: S15-S21Abstract Full Text Full Text PDF PubMed Scopus (216) Google Scholar]. Of the 42 studies included in the IRE study, 88% of them measured knowledge and/or attitudes, yet none of this evidence was featured in the report (one of the 43 studies could not be located). Second, the IRE report did not adhere to standards typical of scientific reviews. For example, the report failed to provide important bibliographic information for the 43 studies. The search strategy and exact criteria used to determine study eligibility for inclusion were not documented. Furthermore, studies were included despite not adhering to the “time” element of the review’s question (i.e., studies were included that did not gather data at 12 months, which was a requirement for program success) and despite failing to collect data on the indicators the authors required for program effectiveness. Third, our reanalysis found that 74% of studies contained one or more discrepancies between the IRE data table and the actual study data. Moreover, the IRE report’s conclusions did not always align with the data they presented. The authors claimed a staggering program failure rate of 89%; however, our reanalysis found that the biased framework and stringent requirements for success caused the body of evidence to be misrepresented. At best, our findings indicate that the IRE analysis lacks the rigor necessary to inform any recommendations on CSE programming, and at worst, the report intentionally downplays the effectiveness of CSE interventions to support the authors’ ideologically driven stance against CSE education. The findings of the IRE report depart markedly from those of other reviews on CSE effectiveness. Several reviews of CSE programs have found compelling evidence in support of the effectiveness and relevance of CSE interventions for young people. We recognize that the implementation of CSE programs is not without limitations. CSE programs are often poorly implemented, lack fidelity, and are seldom delivered at scale [[9]Chandra-Mouli V. Lane C. Wong S. What does not work in adolescent sexual and reproductive health: A review of evidence on interventions commonly accepted as best practices.Glob Health Sci Pract. 2015; 3: 333-340Crossref PubMed Scopus (157) Google Scholar]. CSE curricula often lack important basic information and lack advocacy skill building for young people [[10]Population CouncilSexuality education: A 10-country review of school curricula in East and Southern Africa.https://unesdoc.unesco.org/ark:/48223/pf0000221121Date: 2012Date accessed: October 20, 2021Google Scholar]. Additionally, teachers are frequently ill-prepared and poorly trained to deliver CSE programs [[8]Haberland N. Rogow D. Sexuality education: Emerging trends in evidence and practice.J Adolesc Health. 2015; 56: S15-S21Abstract Full Text Full Text PDF PubMed Scopus (216) Google Scholar]. Therefore, despite proven to be effective, CSE programs tend to flounder when not implemented with adequate fidelity. Still, there is a robust evidence base and decades of programmatic experience that lend support to the benefits of CSE for children and young people. Attempts to undermine the effectiveness of CSE programs threaten the health and well-being of young people across the globe. The IRE authors’ claims that the existing evidence discredits CSE underscore the need for independent replication, a cornerstone of the scientific process. Our reanalysis illuminates the IRE’s biased methods and the inaccuracies of their findings which together compromise the validity of the report’s overall conclusions against the effectiveness of CSE. It showcases the importance of replication in evaluating the robustness, rigor, and potential bias of reviews that claim to speak to the weight of the evidence on a given subject. As the evidence base on CSE grows, the need to conduct reviews and meta-analyses with the goal of informing policy and decision-making heightens, and so does the need to challenge reviews that base their conclusions on tenuous evidence or erroneous claims.