Review this article. How is this particular article relevant to the organization/agency that you have selected as the focus of your final grant proposal for this course? Organization : Planned parenthood DEBATE Open Access Writing implementation research grant proposals: ten key ingredients Enola K Proctor*, Byron J Powell, Ana A Baumann, Ashley M Hamilton and Ryan L Santens Abstract Background: All investigators seeking funding to conduct implementation research face the challenges of preparing a high-quality proposal and demonstrating their capacity to conduct the proposed study. Applicants need to demonstrate the progressive nature of their research agenda and their ability to build cumulatively upon the literature and their own preliminary studies. Because implementation science is an emerging field involving complex and multilevel processes, many investigators may not feel equipped to write competitive proposals, and this concern is pronounced among early stage implementation researchers. Discussion: This article addresses the challenges of preparing grant applications that succeed in the emerging field of dissemination and implementation. We summarize ten ingredients that are important in implementation research grants. For each, we provide examples of how preliminary data, background literature, and narrative detail in the application can strengthen the application. Summary: Every investigator struggles with the challenge of fitting into a page-limited application the research background, methodological detail, and information that can convey the project’s feasibility and likelihood of success. While no application can include a high level of detail about every ingredient, addressing the ten ingredients summarized in this article can help assure reviewers of the significance, feasibility, and impact of the proposed research. Keywords: Implementation research, Grant writing, Preliminary studies Background Investigators seeking funding to conduct implementa- tion research face the challenges of preparing a high- quality proposal and demonstrating their capacity to conduct the proposed study. Researchers need to de- monstrate the progressive nature of their research agenda and their ability to build cumulatively upon the literature and their own preliminary studies. Because implementa- tion science is an emerging field involving complex and multilevel processes, most investigators may feel ‘new to the field.’ Furthermore, young investigators may have less preliminary data, and the path to successful proposal wri- ting may seem less clear. This article identifies ten of the important ingredients in well-crafted implementation proposals; in particular, it addresses how investigators can set the stage for proposed work through pilot data and a well-crafted and rationalized proposed study approach. It addresses ques- tions such as: What preliminary work is important in the grant applications, and how can implementation researchers meet this challenge? How can investigators balance scientific impact with feasibility? Where in an im- plementation research proposal can investigators demon- strate their capacity to conduct a study as proposed? The importance of the question A significant and innovative research question is the first and primary ingredient in a successful proposal. A com- petitive implementation research application needs to pursue scientific questions that remain unanswered, questions whose answers advance knowledge of imple- mentation with generalizability beyond a given setting. By definition, implementation research in health focuses on a health condition or disease, healthcare settings, and particular evidence-based interventions and programs with promise of reducing a gap in quality of care. It is * Correspondence: ekp@wustl.edu Center for Mental Health Services Research, George Warren Brown School of Social Work, Washington University in St. Louis, Campus Box 1196, One Brookings Drive, St. Louis, MO 63130, USA Implementation Science © 2012 Proctor et al.; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Proctor et al. Implementation Science 2012, 7:96 http://www.implementationscience.com/content/7/1/96 mailto:ekp@wustl.edu
http://creativecommons.org/licenses/by/2.0
conducted in usual care settings with practical quality gaps that stakeholders want to reduce. However, to make a compelling argument for scientific innovation and public health significance, a research grant application must have potential beyond reducing a quality gap and implementing a particular evidence-based healthcare practice. The application must have potential to advance the science of implementation by yielding generalizable knowledge. With only one journal devoted solely to im- plementation science [1], researchers must be aware of implementation literature that is scattered across a host of discipline-specific journals. Implementation researchers— akin to students with multiple majors—must demonstrate their grounding in implementation science, health di- seases, disorders and their treatments, and real-world healthcare delivery. Although implementation science is often character- ized as an emerging field, its bar for scientifically impor- tant questions is rising rapidly. Descriptive studies of barriers have dominated implementation science for too long, and the field is urged to ‘move on’ to questions of how and why implementation processes are effective. Accordingly, the Institute of Medicine [2] has identified studies comparing the effectiveness of alternative dis- semination and implementation strategies as a top- quartile priority for comparative effectiveness research. But experimental studies testing implementation stra- tegies need to be informed by systematic background re- search on the contexts and processes of implementation. While investigators must demonstrate their understand- ing of these complexities, their grant proposals must ba- lance feasibility with scientific impact. This paper addresses the challenges of preparing grant applications that succeed on these fronts. Though this article focuses on U.S. funding sources and grant mechanisms, the principles that are discussed should be relevant to imple- mentation researchers internationally. Guidance from grant program announcements Grant review focuses on the significance of proposed aims, impact and innovation, investigator capacity to conduct the study as proposed, and support for the study hypotheses and research design. The entire appli- cation should address these issues. Investigators early in their research careers or new to implementation science often struggle to demonstrate their capacity to conduct the proposed study and the feasibility of the proposed methods. Not all National Institutes of Health (NIH) program announcements require preliminary data. How- ever, those that do are clear that applications must con- vey investigator training and experience, capacity to conduct the study as proposed, and support for the study hypotheses and research design [3]. The more complex the project, the more important it is to provide evidence of capacity and feasibility [4]. The R01grant mechanism is typically large in scope compared to the R03, R21 and R34a. Program announce- ments for grant mechanisms that are preliminary to R01 studies give important clues as to how to set the stage for an R01 and demonstrate feasibility. Investigator ca- pacity can be demonstrated by describing prior work, experience, and training relevant to the application’s set- ting, substantive issues, and methodology—drawing on prior employment and research experience. For example, the NIH R03 small grant mechanism is often used to es- tablish the feasibility of procedures, pilot test instru- ments, and refine data management procedures to be employed in a subsequent R01. The NIH R21 and the R34 mechanisms support the development of new tools or technologies; proof of concept studies; early phases of research that evaluate the feasibility, tolerability, accept- ability and safety of novel treatments; demonstrate the feasibility of recruitment protocols; and support the de- velopment of assessment protocols and manuals for pro- grams and treatments to be tested in subsequent R01 studies. These exploratory grants do not require exten- sive background material or preliminary information, but rather serve as sources for gathering data for subse- quent R01 studies. These grant program announcements provide a long list of how pre-R01 mechanisms can be used, and no single application can or should provide all the stage-setting work exemplified in these descriptions. Review criteria, typically available on funding agency web sites or within program announcements, may vary slightly by funding mechanism. However grants are typ- ically reviewed and scored according to such criteria as: significance, approach (feasibility, appropriateness, ro- bustness), impact, innovation, investigator team, and re- search environment. Table 1 summarizes the ten ingredients, provides a checklist for reviewing applica- tions prior to submission, and ties each ingredient to one or more of the typical grant review criteria. Discussion Approach The literature does not provide a ‘. . . a comprehensive, prescriptive, and robust-yet practical-model to help. . .re- searchers understand (the) factors need to be considered and addressed’ in an R01 study [5]. Therefore we exam- ined a variety of sources to identify recommendations and examples of background work that can strengthen implementation research proposals. This paper reflects our team’s experience with early career implementation researchers, specifically through training programs in implementation science and our work to provide tech- nical assistance in implementation research through our university’s Clinical and Translational Science Award Proctor et al. Implementation Science 2012, 7:96 Page 2 of 13 http://www.implementationscience.com/content/7/1/96 CTSA program. We also studied grant program announcements, notably the R03, R21, R18, and R01 program announcements in implementation science [6-9]. We studied how successful implementation re- search R01 grant applications ‘set the stage’ for the proposed study in various sections of the proposal. We conducted a literature search using combinations of the following key words: ‘implementation research,’ ‘im- plementation studies,’ ‘preliminary studies,’ ‘preliminary data,’ ‘pilot studies,’ ‘pilot data,’ ‘pilot,’ ‘implementation stages,’ ‘implementation phases,’ and ‘feasibility.’ We also drew on published studies describing the introduction and testing of implementation strategies and those that characterize key elements and phases of implementation research [10,11]. From these reviews, we identified ten ingredients that are important in all implementation research grants: the gap between usual care and evidence-based care; the background of the evidence-based treatment to be implemented, its empirical base, and requisites; the theo- retical framework for implementation and explicit theo- retical justification for the choice of implementation strategies; information about stakeholders’ (providers, consumers, policymakers) treatment priorities; the set- ting’s (and providers’) readiness to adopt new treatments; the implementation strategies planned or considered in order to implement evidence-based care; the study team’s experience with the setting, treatment, or implementation process and the research environ- ment; the feasibility and requisites of the proposed methods; the measurement and analysis of study va- riables; and the health delivery setting’s policy/funding environment, leverage or support for sustaining change. Given the sparse literature on the importance of prelim- inary studies for implementation science grant applica- tions, we ‘vetted’ our list of grant application components with a convenience sample of experts. Ultimately, nine experts responded to our request, including six members of the Implementation Science editorial board. We asked the experts to rate the importance of each of the ten ele- ments, rating them as ‘1: Very important to address this is the application,’ ‘2: Helpful but not necessary to the appli- cation,’ or ‘3: Not very important to address’ within the context of demonstrating investigator capacity and study feasibility. Respondents were also asked whether there are any additional factors that were not listed. While all the ten ingredients below were considered important for a successful application, several experts noted that their importance varies according to the aims of the application. For example, one expert affirmed the Table 1 Ten key ingredients for implementation research proposals Proposal ingredient Key question Review criteria Check (yes/no) 1. The care gap or quality gap The proposal has clear evidence that a gap in quality exists? Significance Impact 2. The evidence-based treatment to be implemented Is the evidence for the program, treatment, or set of services to be implemented demonstrated? Significance Innovation 3. Conceptual model and theoretical justification The proposal delineates a clear conceptual framework/theory/ model that informs the design and variables being tested? Approach Innovation 4. Stakeholder priorities, engagement in change Is there a clear engagement process of the stakeholders in place? Significance Impact Approach Environment 5. Setting’s readiness to adopt new services/treatments/programs Is there clear information that reflects the setting’s readiness, capacity, or appetite for change, specifically around adoption of the proposed evidence-based treatment? Impact Approach Environment 6. Implementation strategy/process Are the strategies to implement the intervention clearly defined, and justified conceptually? Significance Impact Innovation 7. Team experience with the setting, treatment, implementation process Does the proposal detail the team’s experience with the study setting, the treatment whose implementation is being studied, and implementation processes? Approach Investigator team 8. Feasibility of proposed research design and methods Does the methods section contain as much detail as possible, as well as lay out possible choice junctures and contingencies, should methods not work as planned? Approach Investigator team 9. Measurement and analysis section Does the proposal clarify the key constructs to be measured, corresponding to the overarching conceptual model or theory? Approach Investigator team Is a measurement plan clear for each construct? Does the analysis section demonstrate how relationships between constructs will be tested? 10. Policy/funding environment; leverage or support for sustaining change Does the proposal address how the implementation initiative aligns with policy trends? Impact Significance Proctor et al. Implementation Science 2012, 7:96 Page 3 of 13 http://www.implementationscience.com/content/7/1/96 importance of the settings’ readiness to change, but noted that it may not be crucial to address in a given proposal: ‘the setting’s readiness may be unimportant to establish or report prior to the study, because the study purpose may be to establish an answer to this question.’ However, another maintained, ‘in a good grant applica- tion, you have to dot all the ‘I’s’ and cross all the ‘T’s.’ I consider all these important.’ One expert noted that applications might need to argue the importance of im- plementation research itself, including the importance of closing or reducing gaps in the quality of care. This was viewed as particularly important when the study section to review the grant may not understand or appreciate implementation research. In these cases, it may be im- portant to define and differentiate implementation re- search from other types of clinical and health services research. For example, it may be useful to situate one’s proposal within the Institute of Medicine’s ‘prevention research cycle,’ which demonstrates the progression from pre-intervention, efficacy, and effectiveness research to dissemination and implementation studies that focus on the adoption, sustainability, and scale-up of interven- tions [12]. It may also be important to convey that im- plementation research is very complex, necessitating the use of multiple methods, a high degree of stakeholder involvement, and a fair amount of flexibility in order to ensure that implementers will be able to respond appro- priately to unforeseen barriers. Ten key ingredients of a competitive implementation research grant application As emphasized at the beginning of this article, the essen- tial ingredient in a successful implementation science proposal is a research question that is innovative and, when answered, can advance the field of implementation science. Assuming that an important question has been established to potential reviewers, we propose that the following ten ingredients can help investigators demon- strate their capacity to conduct the study and to de- monstrate the feasibility of completing the study as proposed. For each ingredient, we provide examples of how preliminary data, background literature, and nar- rative detail in the application can strengthen the application. The care gap, or quality gap, addressed in the application The primary rationale for all implementation efforts, and thus a key driver in implementation science, is discove- ring how to reduce gaps in healthcare access, quality, or, from a public health perspective, reducing the gap be- tween Healthy People 2020 [13] goals and current health status. Accordingly, implementation research proposals should provide clear evidence that gaps exists and that there is room for improvement and impact through the proposed implementation effort. This is a primary way of demonstrating the public health significance of the proposed work. Gaps in the quality of programs, services, and health- care can be measured and documented at the popula- tion-, organization-, and provider-levels [14]. Several kinds of preliminary data can demonstrate the quality gap to be reduced through the proposed implementation effort. For example, investigators can emphasize the bur- den of disease through data that reflect its morbidity, mortality, quality of life, and cost [14]. An implementa- tion research grant should cite service system research that demonstrates unmet need [15], the wide variation in the use of evidence-based treatments in usual care [16-19], or the association between the burden of disease and variations in the use of guidelines [20]. Investigators can also document that few providers adopt evidence- based treatments [21,22], that evidence-based treatments or programs have limited reach [23], or that penetration [24] into a system of care can be addressed by the imple- mentation study. Regardless of the specific approach to documenting a quality gap, investigators should use rigorous methods and involve all relevant stakeholders [14]. In fact, stakeholders can demonstrate their involve- ment and endorse quality gaps through letters of sup- port attesting to the lack of evidence-based services in usual care. The evidence-based treatment to be implemented A second key ingredient in implementation research proposals is the evidence-based program, treatment, pol- icies, or set of services whose implementation will be studied in the proposed research [25-27]. The research ‘pipeline’ [28-30] contains many effective programs and treatments in a backlog, waiting to be implemented. Moreover, many health settings experience a huge de- mand for better care. An appropriate evidence-based treatment contributes to the project’s public health sig- nificance and practical impact, presuming of course that it will be studied in a way that contributes to implemen- tation science. Implementation research proposals must demonstrate that the evidence-based service is ready for implementa- tion. The strength of the empirical evidence for a given guideline or treatment [31,32], a key part of ‘readiness,’ can be demonstrated in a variety of ways; in some fields, specific thresholds must be met before an intervention is deemed ‘evidence-based’ or ‘empirically-supported’ [33-35]. For example, Chambless et al. [35] suggest that inter- ventions should demonstrate efficacy by being shown to be superior to placebos or to another treatment in at least two between group design experiments; or by showing efficacy in a large series of single case design experiments. Further, Chambless et al. [35] note that Proctor et al. Implementation Science 2012, 7:96 Page 4 of 13 http://www.implementationscience.com/content/7/1/96 the experiments must have been conducted with treatment manuals, the characteristics of the samples must have been clearly specified, and the effects must have been demonstrated by at least two different investigators or investigative teams. The strength of evidence for a given treatment can also be classified using the Cochrane EPOC’s criteria for levels of evidence, which considers randomized con- trolled trials, controlled clinical trials, time series designs, and controlled before-and-after studies as ap- propriate [36]. Researchers who come to implementation research as effectiveness researchers or as program or treatment developers are well positioned, because they can point to their prior research as part of their own background work. Other researchers can establish readi- ness for implementation by reviewing evidence for the treatment or program as part of the background literature review, preferably relying on well-conducted systematic reviews and meta-analyses of randomized- controlled trials (if available). At a minimum, ‘evaluability assessment’ [37] can help reflect what changes or improvements are needed to optimize effectiveness given the context of the implementation effort. Conceptual model and theoretical justification Any research striving for generalizable knowledge should be guided by and propose to test conceptual frame- works, models, and theories [38]. Yet, theory has been drastically underutilized and underspecified in imple- mentation research [38-40]. For example, in a review of 235 implementation studies, less than 25% of the studies employed theory in any way, and only 6% were explicitly theory-based [39]. While translating theory into research design is not an easy task [36], the absence of theory in implementation research has limited our ability to spe- cify key contextual variables and to identify the precise mechanisms by which implementation strategies exert their effects. McDonald et al. [41] present a useful hierarchy of theories and models, which serves to organize the differ- ent levels of theory and specify the ways in which they can be useful in implementation research. They differen- tiate between conceptual models, frameworks, and sys- tems, which are used to represent global ideas about a phenomenon and theory, which is an ‘organized, heuris- tic, coherent, and systematic set of statements related to significant questions that are communicated in a mean- ingful whole’ [41]. Within the realm of theory, they dif- ferentiate between grand or macro theories (e.g., Rogers’ Diffusion of Innovations theory [26]), mid-range theories (e.g., transtheoretical model of change [42]), and micro- theories (e.g., feedback intervention theory [43]). Though models, frameworks, and systems are generally at a higher level of abstraction than theories, it is important to note that the level of abstraction varies both between and within the categories of the hierarchy. The thought- ful integration of both conceptual models and theories can substantially strengthen an application. Conceptual models, frameworks, and systems can play a critical role in anchoring a research study theo- retically by portraying the key variables and relation- ships to be tested. Even studies that address only a subset of variables within a conceptual model need to be framed conceptually, so that reviewers perceive the larger context (and body of literature) that a particular study proposes to inform. Given the confusion sur- rounding definitions and terminology within the still- evolving field of dissemination and implementation [44,45], grant proposals need to employ consistent lan- guage, clear definitions for constructs, and the most valid and reliable measures for the constructs that cor- respond to the guiding conceptual framework or theo- retical model. Proposal writers should be cautioned that the theory or conceptual model used to frame the study must be used within the application. A mere mention will not suffice. A conceptual model can help frame study questions and hypotheses, anchor the background literature, clarify the constructs to be mea- sured, and illustrate the relationships to be evaluated or tested. The application must also spell out how potential findings will inform the theory or model. Numerous models and frameworks can inform imple- mentation research. For example, Glasgow et al. [23] RE-AIM framework can inform evaluation efforts in the area of implementation science. Similarly, Proctor et al. [46] have proposed a model that informs eva- luation by differentiating implementation, service sys- tem, and clinical outcomes, and identifying a range of implementation outcomes that can be assessed [24]. Damschroder et al.’s [10] Consolidated Framework for Implementation Research identifies five domains that are critical to successful implementation: intervention characteristics (evidentiary support, relative advantage, adaptability, trialability, and complexity); the outer setting (patient needs and resources, organizational connected- ness, peer pressure, external policy and incentives); the inner setting (structural characteristics, networks and communications, culture, climate, readiness for imple- mentation); the characteristics of the individuals involved (knowledge, self-efficacy, stage of change, identification with organization, etc.); and the process of implementa- tion (planning, engaging, executing, reflecting, evaluating). Others have published stage or phase models of imple- mentation. For example, the Department of Veteran Affairs’ QUERI initiative [47] specifies a four-phase model spanning pilot projects, small clinical trials, regional implementation, and implementation on the national scale; and Aarons, Hurlburt and Horwitz [48] Proctor et al. Implementation Science 2012, 7:96 Page 5 of 13 http://www.implementationscience.com/content/7/1/96 developed a four phase model of exploration, adoption/ preparation, active implementation, and sustainment. Magnabosco [49] delineates between pre-implementa- tion, initial implementation, and sustainability planning phases. McDonald et al. [41] note that grand theories are similar to conceptual models, and that they generally represent theories of change. They differentiate be- tween classical models of change that emphasize na- tural or passive change processes, such as Rogers’ diffusion of innovations theory [26], and planned mo- dels of change that specify central elements of active implementation efforts. Investigators may find it more helpful to draw from mid-range theories because they discuss the mechanisms of change at various levels of the implementation context [26]. For example, social psychological theories, organizational theories, cogni- tive psychology theories, educational theories, and a host of others may be relevant to the proposed pro- ject. While conceptual models are useful in framing a study theoretically and providing a ‘big picture’ of the hypothesized relationships between variables, mid- range theories can be more helpful in justifying the se- lection of specific implementation strategies specifying the mechanisms by which they may exert their effects. Given the different roles that theory can play in imple- mentation research, investigators would be wise to consider relevant theories at multiple levels of the the- oretical hierarchy when preparing their proposals. It is far beyond the scope of this article to review concep- tual models and theories in detail; however, several authors have produced invaluable syntheses of concep- tual models and theories that investigators may find useful [10,41,50-56]. Stakeholder priorities and engagement in change Successful implementation of evidence-based interven- tions largely depends on their fit with the preferences and priorities of those who shape, deliver, and participate in healthcare. Stakeholders in implementation, and thus in implementation research, include treatment or guide- line developers, researchers, administrators, providers, funders, community-based organizations, consumers, families, and perhaps legislators who shape reimburse- ment policies (see Mendel et al.’ article [57] for a frame- work that outlines different levels of stakeholders). These stakeholders are likely to vary in their knowledge, perceptions, and preferences for healthcare. Their per- spectives contribute substantially to the context of im- plementation and must be understood and addressed if the … Read more
Applied Sciences
Architecture and Design
Biology
Business & Finance
Chemistry
Computer Science
Geography
Geology
Education
Engineering
English
Environmental science
Spanish
Government
History
Human Resource Management
Information Systems
Law
Literature
Mathematics
Nursing
Physics
Political Science
Psychology
Reading
Science
Social Science
Home
Blog
Archive
Essay
Reviews
Contact
google+twitterfacebook
Copyright © 2019 HomeworkMarket.com