Examination of Data, Analytical Issues and Proposed Methods for Conducting Comparative Effectiveness Research Using "Real-World Data"

BACKGROUND: The Patient Protection and Affordable Care Act brought considerable attention to comparative effectiveness research (CER). OBJECTIVES: To (a) suggest best practices for conducting and reporting CER using “real-world data” (RWD), (b) describe some of the data and infrastructure requirements for conducting CER using RWD, (c) identify statistical challenges with the analysis of nonrandomized studies and suggest appropriate techniques to address those challenges, (d) recognize the value of patient-reported outcomes in CER, (e) encourage the incorporation of observational data into randomized controlled studies, and (f) highlight the importance of incorporating payers in industry-sponsored research. SUMMARY: The first article in this supplement, “Something old, something new...” provides a policy perspective on the recent evolution of CER. It reviews the historical context, discusses the “promise and fear” of CER, and then describes the new role of the Patient-Centered Outcomes Research Institute (PCORI) in defining and sponsoring CER. The second paper, “Ten Commandments,” proposes a series of tenets for planning, conducting, and reporting CER done with RWD. Oriented for basic-tointermediate researchers, it combines standard scientific research principles with considerations specific to nonrandomized, RWD studies. The third article, “Infrastructure Requirements,” points out that effective use of secondary data requires addressing major methodological and infrastructural issues, including development of analytical tools to readily access and analyze data, formulation of guidelines to enhance quality and transparency, establishment of data standards, and creation of data warehouses that respect the privacy and confidentiality of patients. It identifies gaps that must be filled to address the underlying issues, with emphasis on data standards, data quality assurance, data warehouses, computing environment, and protection of privacy and confidentiality. The fourth paper, “Statistical Issues,” discusses how the validity of analytic results from observational studies is adversely impacted by biases that may be introduced due to lack of randomization. It reviews some of the methodological challenges that arise in the analysis of data from nonrandomized studies, with particular emphasis on the limitations of traditional approaches and potential solutions from recent methodological developments. The fifth paper, “Considerations on the Use of Patient Reported Outcomes (PROs),” describes how PRO data can play a critical role in guiding patients, health care providers, payers, and policy makers in making informed decisions regarding patient-centered treatment from among alternative options and technologies and have been noted as such by PCORI. However, collection and interpretation of such data within the context of CER have not yet been fully established. It discusses some challenges with including PROs in CER initiatives, provides a framework for their effective use, and proposes several areas for future research. Lastly, “Developing a Collaborative Study Protocol...” indicates that there is the potential, the desire, and the capability for payers to be involved in CER studies, combining elements of their own observational data with prospective studies. It describes a case example of a payer, a pharmaceutical company, and a research organization collaborating on a prospective study to examine the effect of prior authorization for pregabalin on health care costs to the payer. CONCLUSION: Researchers at Pfizer routinely conduct CER-type studies. In this supplement, we have proposed some approaches that we believe are useful in developing certain kinds of evidence and have described some of our experiences. Our experiences also make us acutely aware of the limitations of approaches and data sources that have been used for CER studies and suggest that there is a need to further develop methods that are most useful for answering CER questions. J Manag Care Pharm. 2011;17(9-a):S3-S4 Copyright © 2011, Academy of Managed Care Pharmacy. All rights reserved. With the appropriation of funds through the American Recovery and Re-investment Act of 2009 and the passage of the Patient Protection and Affordable Care Act in 2010, there is heightened awareness about the need to explore alternative approaches for comparative effectiveness that extend beyond the paradigm offered by traditional randomized controlled trials (RCTs). One potential approach is the use of “real-world data” (RWD) to supplement traditional RCTs. While the term RWD is not new, there exists a controversy over the use and the meaning of RWD. The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) created a Task Force in 2004 to develop a framework to deal with RWD, and their first task was to define “real-world” data. Even among the members of the task force, considerable debate on the definition of RWD ensued. In the end, the Task Force agreed on the following definition, “[RWD] are data used for decision-making that are not collected in conventional RCTs.”1 While we adhere to ISPOR’s general definition of RWD, our focus in this collection of articles is particularly on studies of nonrandomized data sources (e.g., claims databases) which are usually conducted retrospectively, as well as prospective pragmatic studies. To further elaborate on the uses of RWD in the formulary decision-making process, a group of individuals with experience in pharmacoeconomic and outcomes research and those with experience in managed care formulary decisions convened to explore the perceptions and future of incorporating RWD into decision making.2 That group also recognized the importance of incorporating RWD in the decision-making process. In both aforementioned papers including those included in this supplement, the strengths of traditional RCTs are acknowledged. It is important to note that RWD will never replace the more traditional and more robust RCT data; however, the emerging trend is to incorporate data that are more generalizable. However, as we embark on this new paradigm, it is helpful to reflect upon and learn from past attempts to reshape health care. The first article in this supplement provides a brief history of CER, describes the current state of affairs, and introduces and highlights the importance of the Patient-Centered Outcomes Research Institute (PCORI) and its role in CER methods and evidence generation. With advances in health information technology, payers and researchers have more data on medicines than what pharmaceutical companies have historically been able to provide at the time of a drug product’s launch. With the increasing sophistication of payers to conduct their own research, it is more important than ever to ensure that researchers are equipped with a concise set of best practices of the essentials for conducting CER. Therefore, the second article in this supplement discusses 10 tenets on conducting CER using RWD, which we believe may be used as an “instructional guide” for others