Disseminating the EBP Project Assignment

Disseminating the EBP Project Assignment

Disseminating EBP Projects

Many factors influence how successfully an EBP Project is implemented and evaluated. As noted in this week’s Learning Resources, using a translation framework can facilitate effective implementation and the evaluation of outcomes.
For this assignment you will prepare a scholarly product that reports the results and implications of your project to designed audience.  The scholarly product for dissemination reports the results and implications of your project to a specified audience. Your scholarly product may be in the form of a project summary and evaluation report, poster presentation, program evaluation report, or a media or technology-based deliverable. Disseminating the EBP Project Assignment

BUY A PLAGIARISM-FREE PAPER HERE

To prepare for this week’s section of Application 5:

  • Reflect      on the definitions and criteria for scholarship presented in the Learning      Resources and in your weekly discussions.
  • Consider      AACN’s definition of scholarship and their vision of the role of the      DNP-prepared nurse in furthering nursing scholarship.

The full Application 5 is due by Tuesday 11/06/18 of this week. Instructions for how to prepare for previous sections of this Application have been provided in Weeks 9 and 10. Disseminating the EBP Project Assignment

To complete:

Assignment 5: Disseminating the EBP Project

For this assignment you will prepare a scholarly product that reports the results and implications of your project to designed audience.  The scholarly product for dissemination reports the results and implications of your project to a specified audience. Your scholarly product may be in the form of a project summary and evaluation report,  poster presentation, program evaluation report, or a media or technology-based deliverable, Your scholarly product must include the following: Disseminating the EBP Project Assignment

  • Background,      Purpose, and Nature of the Project (2 points)
    Describe the larger problem or issue you explored in your DNP Project.      Identify your practice-focused question. Explain how this problem or issue      affects nursing practice and the overall health care system, and describe      how your DNP Project explored possible strategies for addressing the      problem or issue.
  • Evidence      (2 points)

Identify the practice setting and context in which you conducted your project, a summary of the evidence supporting the project, and the process for implementation and evaluation. Disseminating the EBP Project Assignment

  • Presentation      of Results (2 points)

Explain the results of your DNP Project and recommendations for change that you made to the organization

Required Readings

White, K. M., Dudley-Brown, S. & Terhan, M.F. (2016). Translation of evidence into nursing and health care practice (2nd ed.) New York, NY: Springer.

Chapter 18, Dissemination of Evidence (SEE ATTACHED FILE).

American Association of Colleges of Nursing. (1999). Defining scholarship for the discipline of nursing. Retrieved from http://www.aacn.nche.edu/publications/position/defining-scholarship

American Association of Colleges of Nursing. (2006). The essentials of doctoral education for advanced nursing practice. Retrieved from http://www.aacn.nche.edu/publications/position/DNPEssentials.pdf

Boyer, E. L. (1992). Scholarship reconsidered: Priorities of the professoriate. Issues in Accounting Education, 7(1), 87-91. Disseminating the EBP Project Assignment

Carina, K., Chan, Y., Oldenburg, B., & Kasisomayajula, V. (2015). Advancing the science of dissemination and implementation in behavioral medicine: Evidence and progress. International Journal of Behavioral Medicine, 22, 277-282 doi 10.1007/s12529-015-9490-2

Glassick, C. E. (2000). Boyer's expanded definitions of scholarship, the standards for assessing scholarship, and the elusiveness of the scholarship of teaching. Academic Medicine, 75(9), 877-880. Retrieved from http://www.academicpeds.org/events/assets/Glassick article.pdf

Lane, J.P. & Stone, V.J. (2015). Comparing three knowledge communication strategies – diffusion, dissemination and translation – through randomized controlled studies. Assistive Technology doi:10.3233/978-1-61499-566-1-92

Leeman, J., Myers, A.E.,Ribisl, K.M.,& Ammerman, A.S. (2015). Disseminating policy and environmental change interventions: Insights from obesity prevention and tobacco control. International Journal of Behavioral Medicine, 22:301–311 doi 10.1007/s12529-014-9427-1

Rieger, K. & Schultz, A.S. (2014). Exploring arts-based knowledge translation: Sharing research findings through performing the patterns, rehearsing the results, staging the synthesis. Worldviews on Evidence-Based Nursing, 11(2), 133–139. Disseminating the EBP Project Assignment

Tricco, a.C., Ashoor, H.M., Cardoso, R., MacDonald, H., Cogo, E., Kastner, M., Perrier, L., McKibbon, A., Grimshaw, J.M. & Straus, S.E. (2016). Sustainability of knowledge translation interventions in healthcare decision making: a scoping review. Implementation Science, 11(55) doi 10.1186/s13012-016-0421-7 Disseminating the EBP Project Assignment

Evaluating EBP Projects

It is the role of nurses to identify practice issues and develop evidence-based solutions to improve the quality of care. One of the issues that affect nursing practice is long intensive care unit (ICU) admission of critical care patients. Extended stay in intensive care is a significant practice problem in health care. There are many risks that face patients who are admitted in ICU for an extended period of time. Longer hospital admission in the ICU has been associated with negative patient outcomes and experiences. The longer patients are admitted in intensive care, the higher the chances of mortality and hospital readmission (Schneider et al., 2012). Disseminating the EBP Project Assignment. Patients’ personal experiences in care are also more negative when they stay in ICU for a long period of time. Therefore, it is important to use EBP solutions to improve the experiences and outcomes of these patients. The proposed solution for this practice issue is the mobilization of patients. Disseminating the EBP Project Assignment. Research shows that early mobilization of critical care patients can have a positive effect on their healthcare experiences and outcomes (Adler and Malone, 2012; Denehy, Lanphere & Needham, 2017; Holdsworth et al., 2015). The purpose of this project is to create a plan for the change to implement this intervention, implement the change, and evaluate the effectiveness of the change to ensure that it has an impact of the improvement of quality and safety of care. Disseminating the EBP Project Assignment

Planning for EBP Change

Even when a new practice approach has been approved for implementation, it may still fail to materialize because of poor management of the change process. It is important to analyze the organization’s readiness for change, deal with the barriers to change, and develop the best framework to implement a given change process (Darling, 2016). The following is an analysis of my organization’s readiness to change and the specific strategies to be used to ensure that the proposed practice change is successful. Disseminating the EBP Project Assignment

Assessing Readiness for Change

There are various factors that either facilitate or limit my organization’s readiness for change. One of the factors that makes the organization ready for change is the great leadership. Disseminating the EBP Project Assignment. Leadership plays a big role in facilitating change. Once the leaders believe in the need for change and are willing to motivate the rest of the organization, it is easier to implement the new practice approach and inspire other people to join in (White, Dudley-Brown & Terharr, 2016). Another facilitator of change in the organization is good communication. Communication is an essential component of a successful change process. Disseminating the EBP Project Assignment. There needs to be good communication of the need and the benefits of the change process so that all the members of the organization can understand the purpose of the change process and be willing to be a part of it,

A major barrier for the change process in this organization is the financial obligations for implementing the new practice standard. There will be a need to invest in a new workforce to facilitate effective mobilization of the patients. There may be a limitation of financial capabilities that may influence the successful implementation of the project. Another barrier is the possibility of resistance. Changing from one practice approach to another may attract resistance because people may want to stick to the usual ways of operations. Disseminating the EBP Project Assignment

Theory/Framework to Guide Change Efforts

The change process for this project will be facilitated by the Theory of Reasoned Action. This theory believes that people’s actions are rational. Behavior is linked to the beliefs, attitudes, and intentions of people towards a given action (White, Dudley-Brown & Terharr, 2016). Therefore, the organization will facilitate the change by developing a model to reason with the employees to show them that changing to the new practice approach is a reasonable thing to do. Disseminating the EBP Project Assignment

The organization will begin by changing the beliefs and attitudes of the workers by showing them evidence of the poor health quality of the ICU patients in the old models of operations. They will later change the intentions of the workers by showing them a new model of operation that they can use to improve the quality of care they provide to the patients. This will help to change their behaviors when helping intensive care patients, leading to the creation of a new norm in the care of these patients. Disseminating the EBP Project Assignment

Strategies to Address Barriers

The strategy to deal with the resistance to change is to have open communication of details pertaining to the new practice approach. It is important that all workers understand the importance of the change process for them to want to participate. As explained by the theory of reasoned action, people are able to change their behaviors when they have altered their attitudes and beliefs towards the change efforts. Disseminating the EBP Project Assignment

Implementing EBP Change

In the process of implementing the EBP project, there are some macro and micro factors that may influence its success. Identifying the desired outcomes and the factors that might affect successful implementation is vital. This helps in the development of mitigation and management strategies that will enhance chances of success. Disseminating the EBP Project Assignment

Desired Outcomes of the EBP Project

There are three main outcomes that will demonstrate a success in this project;

· Improved consistency in patient mobilization

· Reduced lengths of stay

· Reduced patient mortality

· Reduced patient readmissions

After the implementation of this project, it is expected that the mobilization of the patients will be more consistent, with a definite schedule that is understood and used efficiently by the nurses. The effectiveness of the patient mobilization is expected to help reduce the patients’ length of stay in intensive care. The effectiveness of the project will also reduce patient mortality rate and the rates of readmission after being released from the ICU. Disseminating the EBP Project Assignment

Macro or Micro systems Issues inhibiting Implementation

The macro factors that influence the successful implementation of the new practice approach include the unavailability of adequate resources, inadequate staffing, and the lack of knowledge and skills to effectively mobilize the patients as recommended. First, the unavailability of resources could limit the effective implementation of new standards of practice that will help to improve patient safety. Secondly, the standard of mobilizing patients will require more working time for nurses. Disseminating the EBP Project Assignment. There is already a limitation of healthcare professionals. Adding more work will increase the problem of burnouts, which reduce the quality of care. Therefore, there may be a need to add more personnel. Lastly, effective mobilization of patients may require training to be done effectively. Patients in the ICU are quite delicate. It is important that the nurses are equipped with the rights skills to manage such responsibilities. These are all macro factors that need to be handled on the organizational level. Disseminating the EBP Project Assignment

The micro factors are issues at the individual level that can affect the implementation of the new practice approach. According to Maclusky and Middleton (2010), transferring knowledge to practice can be challenging because of factors on the individual level such as personal beliefs and attitudes towards the new practice approach. The micro factors that can affect this project include personal beliefs and attitudes, willingness to improve patient safety, and the professionals’ personal beliefs of their capabilities. Disseminating the EBP Project Assignment. Nurses’ beliefs and attitudes on early mobilization will affect how effectively they perform the intervention. If nurses believe that they are actually making a difference, then they are more likely to participate in the change. The nurses must naturally be willing to engage in the new practice model for effectiveness to be achieved. It is important that professionals believe in their capabilities to make a difference to the patients. They should understand the ways in which they are improving patient safety by practicing the recommended approaches. Disseminating the EBP Project Assignment

Strategies for Resolving those Issues

The only way to achieve success when implementing new evidence-based practice approaches is to develop strategies that deal with the challenges and barriers. The following are the strategies that can be used to deal with the barriers and challenges to this implementation. The first strategy is involving external stakeholders. Implementing this approach will need support from external stakeholders to help with the funding. Disseminating the EBP Project Assignment. There are governmental and non-governmental organizations that can assist in the funding of such a project to enhance patient safety. The second intervention is training and development. Nurses need to be trained to increase understanding and confidence in the practice approach. Training is one of the important strategies required in managing change because employees need to understand the new practice approaches before they can participate. Lastly, open communication is required for the success of this project. There should be open and direct communication of information on the change to increase engagement. Disseminating the EBP Project Assignment

How EBP Project will Improve Quality and Patient Safety

Dealing with the barriers to the implementation of this evidence-based approach will be helpful in improving care for the patients in intensive care. One of the changes that will be enabled is increased consistency in patient mobilization. Once the patients are mobilized more effectively, they will have a faster rate of recovery. Research shows that early mobilization is an effective way of improving the recovery process for the people in intensive care (Denehy, Lanphere & Needham, 2017). Disseminating the EBP Project Assignment. There will also be reduced lengths of stay because of the faster recovery processes. In the end, the patients will end up having better outcomes. The rates of readmission will be reduced significantly. Similarly, this practice approach can help to reduce the mortality rates for the ICU patients. Therefore, it is important to ensure that the barriers to the implementation of this project are effectively dealt with for the benefit of the patients in intensive care. Disseminating the EBP Project Assignment

Evaluating EBP Projects

The outcomes that are to be evaluated in this project include the improved consistency of patient mobilization, increased speed of recovery for ICU patients, and better quality of outcomes, including a reduced rate of mortality and patient readmissions. The following section presents the strategies that would be used to ensure the effective achievement of each of these outcomes. Disseminating the EBP Project Assignment

Evaluating the Outcomes

The outcomes of this project will be examined using qualitative and quantitative approaches. Qualitative evaluation strategies are used to describe the conditions in which the translation project takes place (White, Dudley-Brown & Terharr, 2016). In this case, this form of evaluation will be used to examine the nurses’ experiences in relation to the new practice approach. A descriptive evaluation design will be used to examine the attitudes of the nurses towards the new system and their beliefs on the effects of the new practice approach to their jobs. Disseminating the EBP Project Assignment. This measurement will help to determine how well the implemented strategies have improved the consistency of patient mobilization. For the other outcomes to be measured, a pre and post-measurement strategy in one cohort will be used. This is a measurement of the outcomes in the period that the intervention had not been implemented in comparison to after the implementation (White, Dudley-Brown & Terharr, 2016). This evaluation strategy will help to identify the changes in the speed of recovery of ICU patients, the rates of readmission, and mortality rates. Disseminating the EBP Project Assignment

New Practice Guidelines

Practice guidelines are created after the evaluation of data regarding a given intervention to an identified practice problem. The translation of new evidence into new practice guidelines is essential for the improvement of care quality because it ensures that concerned stakeholders can apply it to enhance patient care (Erickson et al., 2014). Based on the evaluation of the outcomes above, the following are the practice guidelines that I would recommend. Disseminating the EBP Project Assignment. First, I would recommend that patients be mobilized at least once a day in their early stages of recover as long as their fitness has been approved by their physicians. Secondly, I would recommend that patient mobilization be included in the patient treatment records so that it can be easier for the healthcare professionals to track the frequency of the mobilization of the patients. This guideline aims at ensuring that the movement of patients is stable for the achievement of the evaluated outcomes. Disseminating the EBP Project Assignment

New Standards of Care

Standards of care are established criteria that should be met for the patient outcomes to be enhanced. The following are the standards of practice that would be favorable for the enhancement of the care of ICU patients based on the outcomes evaluated. (1) Implementing an early mobility program for the ICU patient with different stages including bed mobility to ambulation, depending on the patient functional ability and as prescribed by the physician (Perme & Chandrashekar, 2009). Disseminating the EBP Project Assignment (2) Team working is essential for the effective implementation of this intervention. It is important that the professionals not only in nursing, but also other areas of work, be cohesive in their approach towards the mobilization of patients. This will reduce the possibility of making errors that can affect the patients’ outcomes (Hopkins & Thomsen, 2007). (3) Another standard of care is to enhance communication. The communication between different professions and the nurses should be of good quality. Specifically, it is recommended that the communication be integrated with the help of electronic health record systems to maximize the efficiency of the performance of the patient mobilization in intensive care (hompson, O’Horo, Pickering & Herasevich, 2015) Disseminating the EBP Project Assignment

Conclusion

Evidence-based practice has become essential in improving patient care safety and quality. However, the successful implementation of new EBP models has been a challenge for many hospitals. This project seeks to implement a new practice approach to deal with the practice problem of extended lengths of stay for ICU patients. Longer admission time has been associated with negative outcomes. Disseminating the EBP Project Assignment. The recommended change to achieve better outcomes for ICU patients is early mobilization, which has been tried but has not been successful due to poor implementation. In this project the factors that can affect the implementation on the micro and macro level have been identified and strategies to counter them developed. To increase the possibility of success, there is a strategy for the evaluation of the achievement of identified outcomes. Both qualitative and quantitative strategies will be used to determine the effectiveness of the change. In return, the quality and safety of care will be improved. Disseminating the EBP Project Assignment

References

Adler, J., & Malone, D. (2012). Early mobilization in the intensive care unit: a systematic review. Cardiopulmonary physical therapy journal, 23(1), 5. Retrieved from: https://eds-a-ebscohost-com.ezp.waldenulibrary.org

Darling, F. (2016). Practitioners’ views and barriers to implementation of the Keeping Birth Normal tool: A pilot study. British Journal of Midwifery, 24(7), 508-519. DOI: https://doi.org/10.12968/bjom.2016.24.7.508

Denehy, L., Lanphere, J., & Needham, D. M. (2017). Ten reasons why ICU patients should be mobilized early. Intensive care medicine, 43(1), 86-90. doi:10.1007/s00134-016-4513-2 Disseminating the EBP Project Assignment

Erickson, K., Monsen, K.A., Artleson, I.S., Radosevich, D.M., Oftedahl, G., Neely, C., &; Thorsen, D.R. (2014) Translation of obesity practice guidelines: Measurement and evaluation. Public Health Nursing, 12(3), 222–23. doi: 10.1111/phn.12169. Disseminating the EBP Project Assignment

Holdsworth, C., Haines, K. J., Francis, J. J., Marshall, A., O’connor, D., & Skinner, E. H. (2015). Mobilization of ventilated patients in the intensive care unit: An elicitation study using the theory of planned behavior. Journal of critical care30(6), 1243-1250. doi: 10.1016/j.jcrc.2015.08.010

hompson, G., O’Horo, J. C., Pickering, B. W., & Herasevich, V. (2015). Impact of the Electronic Medical Record on Mortality, Length of Stay, and Cost in the Hospital and ICU: A Systematic Review and Metaanalysis. Critical Care Medicine43(6), 1276–1282. https://doi-org.ezp.waldenulibrary.org/10.1097/CCM.0000000000000948

Hopkins, R. O., Spuhler, V. J., & Thomsen, G. E. (2007). Transforming ICU culture to facilitate early mobility. Critical care clinics23(1), 81-96. DOI: https://doi.org/10.1016/j.ccc.2006.11.004

Laschinger, H. K. S., & Fida, R. (2014). New nurses’ burnout and workplace wellbeing: The influence of authentic leadership and psychological capital. Burnout Research1(1), 19-28. DOI: https://doi.org/10.1016/j.burn.2014.03.002

McCluskey, A. & Middleton, S. (2010). Delivering an evidence-based outdoor journey intervention to people with stroke: Barriers and enablers experienced by community rehabilitation teams. BMC Health Services Research, 10(18). doi: 10.1186/1472-6962-10-18. Disseminating the EBP Project Assignment

Perme, C., & Chandrashekar, R. (2009). Early mobility and walking program for patients in intensive care units: creating a standard of care. American Journal of Critical Care: An Official Publication, American Association of Critical-Care Nurses18(3), 212–221. https://doi-org.ezp.waldenulibrary.org/10.4037/ajcc2009598

Sadeghi-Bazargani, H., Tabrizi, J.S., & Azami-Aghdash, S. (2014). Barriers to evidence-based medicine: a systematic review. Journal of Evaluation in Clinical Practice, 20, 793-802. DOI: https://doi.org/10.1111/jep.12222

Schneider, E. B., Hyder, O., Brooke, B. S., Efron, J., Cameron, J. L., Edil, B. H., ... & Pawlik, T. M. (2012). Patient readmission and mortality after colorectal surgery for colon cancer: impact of length of stay relative to other clinical factors. Journal of the American College of Surgeons, 214(4), 390-398. doi: 10.1016/j.jamcollsurg.2011.12.025

White, K. M., Dudley-Brown, S., & Terharr, M. F. (2016). Translation of evidence into nursing and health care practice (2nd ed.). New York, NY: Springer.

Assignment 5: Disseminating the EBP Project Grading Rubric

For this assignment you will prepare a scholarly product that reports the results and implications of your project to designed audience. The scholarly product for dissemination reports the results and implications of your project to a specified audience. Your scholarly product may be in the form of a project summary and evaluation report, poster presentation, program evaluation report, or a media or technology-based deliverable; your scholarly product must include the following: Disseminating the EBP Project Assignment

Background, Purpose, and Nature of the Project (2 points)

· Describe the larger problem or issue you explored in your EBP project. Identify your practice-focused question.

· Explain how this problem or issue affects nursing practice and the overall health care system and

· Describe how your EBP project explored possible strategies for addressing the problem or issue. Disseminating the EBP Project Assignment

· Evidence (2 points)

· Identify the practice setting and context in which you conducted your project.

· The summary of the evidence supporting the project.

· The process for implementation.

· The Evaluation. Disseminating the EBP Project Assignment

· Presentation of Results (2 points)

· Explain the results of your EBP project

· The recommendations for change that you made to the organization.

Stage 3: Writing an Evaluation Report

Table of Contents SECTION 1: Executive Summary

SECTION 2: Program Description

SECTION 3: Evaluation Methodology

SECTION 4: Findings

SECTION 5: Interpretation and Reflection SECTION 6: Recommendations

This stage of the guide provides a suggested format for your final evaluation report, along with ideas for developing each section of the report. While this stage covers the basic content for any evaluation report, your report may vary from the model presented here. How you decide to organize your report will depend on the purpose of the report, your audience, and the requirements of the funding agency. (See Appendix H for alternative evaluation report formats.) Disseminating the EBP Project Assignment

51

An “Executive Summary” (sometimes called a Summary) is a short document of one or two pages that appears at the beginning of the final evaluation report. The Executive Summary provides an overview of the program and highlights key findings and recommendations from the evaluation, giving the reader a sense of the report’s content without having to read the entire document. Disseminating the EBP Project Assignment

Why write an Executive Summary? • The Executive Summary outlines what the reader should expect

to find in the report.

• A Summary may be used separately from the report. For instance, it may serve as an efficient means of sharing key findings of the evaluation with a large audience or a potential funder. Disseminating the EBP Project Assignment

What is included in the Executive Summary?

BUY A PLAGIARISM-FREE PAPER HERE

The exact length and components of a summary may vary depending on the purpose of the report. Typically, an executive summary is an overview of the report and includes:

• Purpose of the program • Program activities, setting and population served • Purpose of the evaluation • Overview of findings or outcomes • Overview of recommendations

Tips for writing an Executive Summary • Do not include technical details in the Executive Summary. That

is, do not include details about data collection methods used. • Write the Executive Summary last, after all other sections of the

report are completed. • Write the Executive Summary in a way that allows the reader to

learn about the most salient aspects of the evaluation without reading the full report. Disseminating the EBP Project Assignment

Notes

SECTION 1: Executive Summary

52

The “Program Description” section introduces readers to your program. It should contain a succinct description of the program being evaluated, present program goals and objectives, and explain how program activities were intended to meet the objectives. Depending on your audience, report requirements, and whether you have comparative information about the program from a previous evaluation, you may choose to include additional information about your program and its history in this section. Disseminating the EBP Project Assignment

What is included in the Program Description section?

• Explanation of how the program originated: Provide the rationale for the program in relation to the agency’s mission, research literature, community needs assessment, and/or the political climate.

• Program overview: Focus on the program’s purpose and key program activities. Describe the program’s target population (who is served by the program), when and where activities took place, and why the program was set up the way it was (program design). Disseminating the EBP Project Assignment

• Program goals and objectives: List the program’s goals and objectives.

• Significant program revisions: Describe any changes to the program’s objectives or activities that occurred prior to or during the evaluation, and provide a rationale for those changes.

Additional information to consider including in the Program Description section:

• Relationship of this program to CWIT’s mission and broader organizational efforts.

• History of the program’s development or changes in the program since its initial implementation, including prior accomplishments the current program builds on or gaps its seeks to address. This is especially relevant for programs that have been in existence for several years, and/or if the program received funding from the same agency in prior years. Disseminating the EBP Project Assignment

• Comparison of the program evaluated to similar programs sponsored by CWIT or other agencies.

Notes

SECTION 2: Program Description

53

The “Evaluation Methodology” section of your final evaluation report describes the research methods you used in the evaluation and describes why those particular methods were the most appropriate for the evaluation. The purpose of this section is to explain how the evaluation was designed and carried out and to let your reader assess the quality of your evaluation design. Disseminating the EBP Project Assignment

Why include an Evaluation Methodology section in your final report?

A clear description and justification of the evaluation methods used has several advantages for the program staff and other stakeholders:

• It demonstrates that the evaluation and procedures for collecting data were carefully and systematically planned.

• It tells readers how the evaluation team gathered the information presented in the report. This allows readers to assess the quality of data-collection procedures.

• It provides documentation that program staff can use to repeat procedures if they want to collect comparable data in the future.

• It documents your methods, providing a framework for staff with similar programs to draw on as they design or improve their evaluation procedures.

• It assesses whether or not the data collection tools used were appropriate for the group served by the program.

Remember You may have already described the your evaluation methods in another document, such as a grant application. If so, you may be able to draw on that text for your final report, editing it to reflect any changes in methods used and challenges faced. Disseminating the EBP Project Assignment

Notes

SECTION 3: Evaluation Methodology

54

What should be included in the Evaluation Methodology section of an evaluation report?

• Types of information collected Did you collect quantitative data? If so, what types and why? Did you collect qualitative data? If so, what types and why? Did you collect both kinds of data? If so, what types and why?

• How and why information was collected Describe the data collection tools used and include examples as

appendices. Explain whether the data collection tools existed prior to the

evaluation or if they were developed in-house or by an outside evaluator.

Explain how the data collection tools were intended to answer the research questions.

• Who information was collected from and how participants were selected to provide information

Was information collected from program participants? Program staff? Community members? Other stakeholders? How and why were individuals chosen to provide information for the evaluation? Or did they volunteer? (To review sampling, see Appendix B)

Was a comparison group used? How was the group chosen? (For a review of comparison groups, see Appendix C)

• Who collected the information Was the information collected by program staff? By outside

evaluators? By program participants? Disseminating the EBP Project Assignment

• Limitations in the evaluation design or implementation Were you able to collect the information you needed using your

data collection tools? Were the data collection tools appropriate for those who provided

information? What challenges did you face in carrying out the evaluation? What changes did you make in your evaluation methods over the

course of the project?

Tips for writing your Evaluation Methodology section

• It is not necessary to reproduce the data collection instruments you used in the main text of your evaluation report. However, you may want to include them as appendices. Disseminating the EBP Project Assignment

• The Methodology section of your final evaluation report is important, but need not be long. Be concise!

Notes

55

The "Findings" section presents an organized summary of the information you have collected in a way that describes whether and how well each program objective has been met. In a sense the “Findings” section is the core of the evaluation report, because this is the section in which you provide a coherent account of what you have learned about your program from the data you collected. Disseminating the EBP Project Assignment

How do I “present” my “findings”? Presenting the findings of your evaluation includes organizing the data you have collected, analyzing it, and then describing the results. The first step in presenting your findings is to organize the quantitative and qualitative information you have collected in a way that will allow you to address your objectives. After it is organized, the information can be used to evaluate your objectives. The second step is analyzing the data. “Data analysis” sounds daunting, but all it really means is determining what the information you collected can tell you about whether and how well you met your program objectives. Disseminating the EBP Project Assignment. This entails using the information you have collected to answer your research questions. The third step is describing the analysis of your findings. Data should be presented in condensed, summary form. For example, to describe quantitative data you can use frequencies, percentages or averages. To present qualitative data you might summarize patterns you detected in observational records or clusters of answers in interview responses. Try to avoid interpreting your findings in this section. Interpretations of the findings are comments about why the objectives were or were not met. Interpretations will be presented in the next section of your report. (See Section 5 of this Stage for more information on interpretation.) Disseminating the EBP Project Assignment

Remember It may be tempting to display data in the form of charts, tables or quotations. Be sure to do so only if it will make the results easier for the reader to understand.

Notes

SECTION 4: Findings

56

Example The following is an example of how data may be summarized and analyzed for presentation in the Findings section of a final report. Objective: “60% of women completing the training program during program year 2001 will have acquired the skills need to pass the carpenter’s entrance exam.” Research questions: 1. What percentage of women who completed the training program

passed the carpenter’s entrance exam? 2. After the training program, did women have more skills to pass

the carpenter’s entrance exam than before the program? 3. Did participating women receive the training they expected? Data collected: Disseminating the EBP Project Assignment

• Pre- and post-training assessments of participants’ skills to pass the carpenter’s entrance exam.

• Pre-training focus groups with participants to determine their expectations for the training and assess skill levels.

• Post-training focus groups with participants to assess satisfaction with training, whether expectations were met, and whether skills were obtained.

Presentation of Findings: 1. To answer the first research question, provide the number of

training participants. Present the percentages of participants that did and did not pass the carpenter’s entrance exam.Disseminating the EBP Project Assignment

2. To answer the second research question, summarize the results of pre-training and post-training skills assessments. Describe how pre-training results differed from post-training results. In addition, use qualitative data from focus groups to present participants’ views about whether they felt prepared to pass the exam based on what they learned in the training. Disseminating the EBP Project Assignment

3. To answer the third research question, summarize qualitative responses from focus groups that assess whether the training met participants’ expectations.

Notes

57

Steps for presenting evaluation data 1. Review your program objectives and research questions to

determine how the information you have collected will be used. Think about the types of information you will need to summarize to determine whether or not you have met your objectives.

• Make sure notes taken during interviews and focus groups have been transcribed, and surveys have been entered into a computer database, if appropriate.

2. Read through the data you have collected to be sure it is

understandable and to determine whether any information is missing. Disseminating the EBP Project Assignment

• When you are checking for missing data, keep in mind that missing data may be physical data such as attendance sheets or meeting minutes, or specific information that is needed to evaluate specific objectives.

• If information is unclear, determine whether clarification or additional information is necessary, or whether the information you have should not be included in the report.

• If information is incomplete, determine whether you can and should obtain the missing data.

• Determine how missing data will affect your results. Disseminating the EBP Project Assignment

3. Analyze data and organize analyzed data in the form of charts, lists, graphs or summaries that can be used to report on each objective.

• Address each program objective in your Findings section. If there are insufficient data to determine whether or not the objective was met, then indicate that this was the case and account for the lack of data.

• A conventional method of reporting findings is to state the objective and present data pertaining to the objective immediately below. However, you may also report on clusters of related objectives, particularly process objectives.

• Make sure data are presented clearly. Share your "Findings" section with co-workers to make sure the findings are presented clearly, without interpretation, and in a way that is honest, concise, and complete. Disseminating the EBP Project Assignment

Notes

58

Tips for analyzing qualitative data Qualitative data may be used to answer research questions that require in-depth answers. Examples are questions about individual experiences, program processes, or the relationship between program outcomes and the wider context in which the program took place. Analyzing qualitative data involves looking for patterns, categories, and themes. When examining qualitative data, it is important to put aside what you already know, and allow themes to emerge from the data on their own. Be careful: if you only focus on what you already know to be true about the program and what you have learned in prior evaluations or from personal experience working with the program you may not see other patterns that emerge. Disseminating the EBP Project Assignment

Strategies for analyzing qualitative data: • Content analysis: Look for themes that emerge from data such

as interview or focus group responses, and organize responses according to themes. Themes should relate to your research questions. Be cautious about using interview or focus group questions as themes for organizing data because they are topics imposed on the discussion by those conducting the evaluation, and may not have been discussed otherwise. You may find it useful to ask a co-worker to organize your data according to the themes you have developed. Disseminating the EBP Project Assignment. Compare how each of you organized the responses to ensure they were categorized in a similar way. Lack of agreement may indicate different understandings of the data. This does not mean that one of you was wrong, but rather that you should have a discussion the data and the themes you have generated to come to a common understanding. Once you’ve organized the data into categories or themes, write a statement or short paragraph that summarizes all the responses in each category. Disseminating the EBP Project Assignment

• Case Studies: If you have collected in-depth information about aspects of the program, you may wish to create summary case records from the extensive documentation, and then write descriptive accounts of each person, company, program, or initiative.

Notes

59 Disseminating the EBP Project Assignment

Tips for analyzing quantitative data Quantitative data may be used to answer numerical research questions, such as questions about attendance, retention, test results, and comparisons with a control group. Statistical techniques may be used to convert responses from large surveys or other instruments into averages, frequencies, and percentages, and may also be used to calculate simple correlation. Obtaining expert assistance may be necessary when using advanced statistical techniques. Quantitative data may be tallied by hand or by computer. Disseminating the EBP Project Assignment. Hand tallying involves going through each data record and counting the information you are seeking, and is appropriate when total numbers are small. Computers may be useful for compiling data from surveys or long forms. This involves entering responses using a spreadsheet program such as Excel. Staff members who plan to manage data using a computer should be familiar with spreadsheet programs or obtain support from someone with experience managing quantitative data. Disseminating the EBP Project Assignment

Strategies for analyzing quantitative data: • Frequencies: Frequencies are counts of the number of times

an event occurred. For example, frequencies can be used to count the number of times an answer was given on a survey, the number of people who attended a meeting, or the number of people who completed a training program.

• Means: Means are averages that allow you to describe a group of people or items using a single value. For example, you may wish to calculate the average attendance of a monthly meeting over the course of a year. Averages may also be used to describe salaries of program graduates or numbers of women enrolled in partnering training programs.

• Percentages: Percentages can be used to illustrate how part of a group compared to a whole. For example, you may wish to determine the percentage of trainees who were placed in manufacturing jobs, or the percentage of graduates who stay in their placements for a determined amount of time. Disseminating the EBP Project Assignment

Notes

60

The “Interpretation and Reflection” section of your final evaluation report is where you give meaning to the information you have collected, relating your program’s successes and challenges to factors within and outside of the program and organization.

What is interpretation? Interpretation is a step in the evaluation process in which you make sense of the findings you have presented in the previous section. The interpretation section is a narrative account that gives meaning to the information you have collected. In a sense, you are telling a story with the data in which you draw on your findings to explain the program’s successes and challenges in meeting its objectives. The Interpretations section of your evaluation report includes conclusions about the program that you draw based on the data presented in the Findings section of your evaluation report. When making interpretations for your evaluation report: Disseminating the EBP Project Assignment

• Determine what your results say about your program. Compare results obtained with what was expected.

• Discuss which objectives were and were not met and give possible reasons why.

• Determine whether circumstances or events unrelated to the program contributed to or limited its success.

• Consider how different stakeholders would interpret the results. • Discuss whether different data sources yielded different results,

why this might be the case, and the implications for the evaluation. Disseminating the EBP Project Assignment

• Discuss unexpected findings, including anything learned in the evaluation that was relevant to the program but not directly related to the objectives. These types of interpretations may be useful in determining how the program may be changed in the future.

• Compare your results with results from previous evaluations of your program or similar programs.

• Determine whether there is anything you would like to know about your program that your results do not tell you. This may help you to understand the limitations to your evaluation, and assist in developing research questions for an evaluation in the future. Disseminating the EBP Project Assignment

Notes

SECTION 5: Interpretation and Reflection

61

What are reflections and how do they differ from interpretation?

Reflection is a phase in the evaluation process in which you step back from a narrow focus on the program’s objectives and use your data to examine the relationship between the program and the broader context in which it has taken place. The Reflections section is a place to relate the program’s successes or challenges to factors outside the program itself. You may choose to consider the broader social, political, economic, or cultural issues that had an impact on the program. While most funders do not require a Reflections section, reflection allows you to think beyond your interpretations – that is, to put them in a broader context in order to give them more significance. Reflecting on the evaluation may lead to a better understanding of the program and how the program may be improved in the future. Disseminating the EBP Project Assignment

Suggestions for reflecting on your findings • Set program objectives and data aside and think about what you have

learned. Think about the program, what happened, what worked well, how it could have worked better.

• Reflect alone and also with other staff members and program participants for multiple insights about the program.

• Consider whether there are particular program experiences or stories you can tell that might help explain the program’s successes and challenges by linking them to issues beyond the program itself.

• Be creative. Reflections allow you to move beyond the data to make connections between the program and the wider context in which it took place. Disseminating the EBP Project Assignment

Tips for interpreting and reflecting

• Interpretations and reflections should be presented honestly, in a way that will be helpful to the program. The program will not benefit if you present program strengths and successes with no attention to weaknesses or challenges. The aim is to draw on your data to account for both successes and challenges. This will allow you make meaningful recommendations, and to use the evaluation to continue improving your program. Disseminating the EBP Project Assignment

• Data can be interpreted in many different ways. As with your Findings section, it is often helpful to confer with co-workers and other stakeholders such as program participants about your interpretations and reflections. This will help to ensure the Interpretation and Reflection section is presented clearly, carefully, and honestly. Disseminating the EBP Project Assignment

Notes

62

The “Recommendations” section of your final evaluation report is a narrative list of suggestions for modifying or supplementing the program in the future in order to improve its ability to meet its objectives and increase its success. As you list your recommendations, it is useful to explain the basis for each recommendation and why you believe a particular recommendation will improve the program. Disseminating the EBP Project Assignment

Tips for developing recommendations • Recommendations should draw di directly on what you have

learned in your evaluation and what you know about the program. They should reflect what was reported in your findings and interpretations sections.

• Recommendations may be directed toward the specific program being evaluated. They may also be directed toward the organization, the community being served, the funding agency, or other stakeholders.

• Think about the objectives you did not meet. Are there program

changes that would help you to meet the objectives? Were the objectives unrealistic? Recommendations can suggest changing how the program is organized or staffed; they can also suggest the modification of objectives. Disseminating the EBP Project Assignment

• Recommendations can be creative, as long as they can be realistically implemented.

• Program staff and other stakeholders may be able to provide useful recommendations for the program, or may be helpful in determining whether recommendations are realistic. Disseminating the EBP Project Assignment

• Focus on a reasonable number of recommendations that will have a significant impact on the program. Avoid listing numerous recommendations that require minor changes or that have no context.

• Be specific in explaining how implementing the recommendation will improve the program.

Notes

SECTION 6: Recommendations

Disseminating the EBP Project Assignment