1

DNP Scholarly Program Evaluation

School of Nursing, Northern Kentucky University

DNP 806: Program Development, Implementation, & Evaluation

Dr. Teresa Huber

September 20, 2020

2

DNP Scholarly Program Evaluation

Doctor of Nursing Practice (DNP) scholarly projects help to lead change and improve

patient outcomes within the healthcare industry. It is important to evaluate DNP scholarly

projects in order to determine if the project achieved expected outcomes. This paper uses the

goal-based model to evaluate the effectiveness of a care-team led patient mobility program. This

paper provides an evaluation of objectives, identifies how the financial aspect of the project will

be evaluated for effectiveness, and explains how data collected will be analyzed and how results

will be utilized.

Evaluation Model—Goal-Based Model

Program evaluation is an important part of completing a DNP scholarly project. Terhaar

& Sylvia (2015) explain that a project evaluation helps to support translation into practice,

replication, and dissemination. When evaluating programs, it is important to determine the

specific model for evaluation. The DNP scholarly project for using a care-team led mobility

program will be evaluated using a goal-based model.

Description of Goal-Based Model of Evaluation

The goal-based model of evaluation uses specific metrics to evaluate the effectiveness of

goals and objectives, and this model helps to identify if the intended goals are met or not met

(Pell Institute and Pathways to College Network, 2020). This model helps to answer questions

such as: 1) Did the program achieve intended outcomes? 2) What effect does the program have

on individuals involved? 3) Does the program impact individuals and populations on a larger

scale? (Pell Institute and Pathways to College Network, 2020). Furthermore, using this model of

evaluation helps to determine if the program needs modifications or improvements prior to

disseminating (Pell Institute and Pathways to College Network, 2020).

3

Rationale for Using Goal-Based Model of Evaluation

The goal-based model of evaluation is the best method to use for evaluating the

program’s goals for decreasing length of stay (LOS) and the increasing patient experience.

Inconsistency with using systematic evaluation tools is a common barrier for effectively

measuring the impact of quality improvement projects (Izumi, 2012). The rationale for using a

goal-based model of evaluation is to decrease inconsistencies for evaluating the program’s

effectiveness by providing quantitative data to support the need for continuing or modifying the

program. The goal-based model of evaluation is also the preferred evaluation method for the

hospital system where this project will be completed. Meeting or exceeding the expected goals

will support dissemination across the entire system.

Evaluation of Objectives

The purpose of well-designed DNP projects is to identify and evaluate measurable

outcomes that support EBP and nursing inquiry to improve patient care (Harris, 2020). Harris

(2020) explains that projects should identify key objectives and outcomes that determine the

level of success for executing and disseminating practice changes. The overall goals for

implementation of a care-team led mobility program are to decrease LOS and improve the

patient experience. Read further to learn more about these goals and the evaluation of objectives

associated with each goal.

Goal #1—Decrease LOS

One goal for implementing a care-team led mobility program is to decrease LOS by 0.3

days in the medical-surgical department performing the project. The goal itself is evaluated

based on comparative data. Pre-implementation data will include the previous 6-month average

LOS prior to implementation, and post-implementation data will include the 3-month average

4

LOS after implementation. The objectives to effectively meet this goal are as follows: 1)

Implement the usage of the Johns Hopkins Highest Level of Mobility (JH-HLM) tool; 2) Ensure

compliance with usage of the tool; 3) Educate staff on the steps to take if mobility declines.

Objective #1—Implement Usage of JH-HLM Tool

The JH-HLM tool is an EBP assessment that helps to identify the level of mobility for

patients in the hospital. The tool has an interrater reliability of 0.99 between physical therapists

and nurses (Hoyer et al., 2018). The process for implementing the usage of this tool will involve

in-person education during shift starters with all staff for a period of one month. Nurses and

patient care associates will be provided a copy of the tool and open discussions will be

encouraged to facilitate learning. Furthermore, the assistant nurse manager (ANM) and the nurse

manager (NM) will work alongside nursing staff and patient care associates when using the tool

for one month. This objective will also be evaluated by the ANM and NM on a daily basis in

real-time. Common questions and concerns voiced from staff will be collected, and needed

clarifications will be disseminated to staff.

Objective #2—Compliance with Usage of Tool

Compliance with using the JH-HLM tool will be monitored during department rapid

rounds which are completed by the nursing team, care management, and department hospitalist.

Patient mobility will be discussed using the JH-HLM tool, and all nurses will provide the score

according to the assessment. Compliance with using the tool will be identified during rapid

rounds, and an overall percentage of compliance will be calculated daily, monthly, and quarterly.

Appendix A, Data Collection and Evaluation Tools provides the instrument used to audit

compliance.

5

Objective #3—Steps to Take for Declining Mobility

Some patients may decline in their mobility while hospitalized, and using the JH-HLM

tool will help to identify the level of decline. When this occurs, the first step is to communicate

the declining mobility to the ANM, care manager, and physician. The physician and care

management team should begin planning for subacute rehab placement as soon as a mobility

decline is identified, and physical therapy should be consulted for a thorough mobility

assessment. Proactively working on placement sooner will decrease the patient’s overall LOS.

The reporting of declined mobility will be evaluated on a case-by-case basis using an apparent

cause analysis (ACA) if a decline in mobility was missed and/or not reported.

Goal #2—Improve Patient Experience

The second goal for implementing a care-team led mobility program is to improve the

patient experience as evidenced by a 10% increase for overall rating of the hospital stay and a

10% increase for likelihood to recommend the hospital 90-days post implementation. The

comparative data will consist of the previous 6-month average score for overall rating and

likelihood to recommend. The objectives to effectively meet this goal are as follows: 1)

Communicate the purpose of the mobility program to patients; 2) Ensure mobility level is

maintained or improved.

Objective #1—Communication of Program Purpose to Patients

Patients are the most important stakeholder for this program. Patients need to understand

their level of mobility according to the JH-HML assessment tool. Nurses will need to explain the

benefits of maintaining and/or improving mobility while in the hospital. Evidence shows the

benefits for using a mobility program include increased patient mobility/ambulation, decreased

LOS, increased patient experience, and decreased risk for healthcare acquired conditions (Dewitt

6

et al., 2019; Drolet et al., 2013; Jones et al., 2020; Teodoro et al., 2016; Liu et al., 2018). To

evaluate this objective, patient teach-back will be used to determine if the patient can identify

their current level of mobility and the benefits of maintaining this level of mobility.

Objective #2—Ensure Mobility Level Maintained or Improved

The second objective for this goal is to ensure the patient’s mobility level is maintained

or improved while in the hospital setting. This objective helps to support a positive patient

experience because there is no decline in mobility; therefore, the patient leaves the hospital at the

same level of mobility or better. This objective will be evaluated on a daily basis using the JH-

HLM tool. The tool will be collected daily to determine the percentage of patients who

maintained their current mobility level, percentage of patients who improved their mobility level,

and percentage of patients who declined in mobility. Appendix A, Data Collection and

Evaluation Tools provides the tool used to help analyze the data related to this objective.

Evaluation of Financial Aspect Using a Cost-Effectiveness Analysis

This DNP scholarly project does have some associated costs. The projected overall cost

for implementing the mobility program is $1,208.30. This cost includes expenses such as labor,

paper, ink, card stock, and required training. The financial aspect of this program will be

evaluated using a cost-effectiveness analysis that shows how decreased LOS using a mobility

program leads to decreased costs, increased patient experience, and increased reimbursement.

The cost-effectiveness of the project outweighs the initial $1,208.30 required to start the

program.

Appendix B, Ten Year Comparison of Cost of Hospital Stay and LOS shows an upward

trend for the cost per stay for all inpatient admissions while the LOS for all inpatient admissions

stays the same (Agency for Healthcare Research and Quality, 2020). Based on this data analysis,

7

the focus needs to be on decreasing LOS in order to offset the increased costs because many

insurance companies and the Centers for Medicare and Medicaid Services (CMS) use an

inpatient prospective payment system (IPPS). IPPS is a system that provides a set payment for

specific diagnostic readiness groups (DRG), and it takes into account comorbidities, primary

diagnosis, and secondary diagnoses (Centers for Medicare and Medicaid Services, 2020).

Patients who decline in mobility increase their LOS, and using a mobility program to maintain

and/or improve mobility will decrease LOS. The financial aspects and upfront costs for

implementing a mobility program will be evaluated and proven effective when LOS decreases

which further decreases the cost of care using an IPPS. Patients will be in the hospital a shorter

length of time and payment from CMS and other insurance companies will be the same;

therefore, reimbursement is increased based on the overall decreased LOS.

The cost effectiveness of using a care-team led mobility program will also be evaluated

from a patient experience standpoint. The literature supports that patient experience improves as

mobility is maintained and LOS decreases (Dewitt et al., 2019). Patient experience results are

25% of the hospital’s Value-Based Purchase Score which is used to determine the amount of

reimbursement provided for a hospital stay (Centers for Medicare and Medicaid Services, 2019).

CMS expects for hospitals to improve their patient experience results in order to continue getting

reimbursed for this portion of the overall Value-Based Purchase Score (Centers for Medicare and

Medicaid Services, 2019). Therefore, increasing patient experience scores will help to maintain

and/or increase reimbursement from CMS.

Analysis of Data and Utilization of Results

It is important to analyze data and determine how to utilize results from the DNP

scholarly project. The tools provided in Appendix A, Data Collection and Evaluation Tools will

8

help to quantify results from the project. The data will be analyzed and utilized based on the

primary goals of decreasing LOS and improving the patient experience.

LOS Data Analysis and Utilization

The data analysis for the goal of improving LOS will involve comparing pre-

implementation data from the previous 6-months to data 90-days post-implementation of the

care-team led mobility program. Data analysis that finds at least a 0.3 day improvement in LOS

will support further utilization of the mobility program across the entire hospital system. The

data collected related to the objective for compliance with using the JH-HLM tool will be

analyzed to determine how often nurses are using the tool. The expectation is to collect this data

for a period of 90-days and to become at least 90% compliant with using the JH-HLM tool

during this timeframe. It is important to analyze this compliance objective because staff must use

the tool in order to effectively meet the expectations of the mobility program. The compliance

audit tool is located in Appendix A, Data Collection and Evaluation Tools.

Patient Experience Data Analysis and Utilization

The data analysis for the goal of improving the patient experience will involve comparing

pre-implementation data from the previous 6-months to data 90-days post-implementation of the

care-team led mobility program. Data analysis that finds an improvement of at least 10% for the

overall rating of the hospital and likelihood to recommend will support further utilization of the

mobility program across the hospital system. It is possible that both of these indicators will not

improve equally. If this occurs, a contingency plan can be discussed as it relates to disseminating

the program. Appendix A, Data Collection and Evaluation Tools provides the instruments used

to collect and analyze monthly data related to the patient experience goal and objectives.

9

Conclusion

All in all, the evaluation of the program goals and objectives for using a care-team led

mobility program are based on the evidence related to improving LOS and improving the patient

experience (Dewitt et al., 2019; Lui et al., 2018). This paper provides a program evaluation using

a goal-based model to determine the effectiveness of using a care-team led mobility program.

The program evaluation explains how objectives are evaluated, describes how the financial

aspects of the project are evaluated, and identifies how data is analyzed and results are utilized.

10

References

Agency for Healthcare Research and Quality. (2020). HCUP fast stats—Trends in inpatient

stays. Retrieved from https://www.hcup-

us.ahrq.gov/faststats/NationalTrendsServlet?measure1=02&characteristic1=01&time1=1

0&measure2=04&characteristic2=01&time2=10&expansionInfoState=hide&dataTablesS

tate=hide&definitionsState=hide&exportState=hide

Centers for Medicare and Medicaid Services. (2019). HCAHPS fact sheet. Retrieved from

https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-

Instruments/HospitalQualityInits/HospitalHCAHPS

Centers for Medicare and Medicaid Services. (2020). Medicare learning network: Acute care

hospital inpatient prospective payment system. Retrieved from

https://www.cms.gov/Outreach-and-Education/Medicare-Learning-Network-

MLN/MLNProducts/Downloads/AcutePaymtSysfctshtTextOnly.pdf

Dewitt, K., Coto, J.A., Carr, L., Ondrey, M., & Petkunas, H. (2019). Ambulation programs:

Decreasing length of stay and improving patient outcomes. MEDSURG Nursing, 28(5),

293-302. Retrieved from

http://web.a.ebscohost.com/ehost/pdfviewer/pdfviewer?vid=36&sid=98f2858a-e8aa-

4dff-b734-e9cf9dc3e831%40sdc-v-sessmgr02

Drolet, A., DeJullio, P., Harkless, S., Henricks, S., Kamin, E., Leddy, E., Lloyd, J., Waters, C., &

Williams, S. (2013). Move to improve: The feasibility of using an early mobility protocol

to increase ambulation in the intensive care and intermediate care settings. Physical

Therapy, 93(2), 197-207. https://doi.org/10.2522/ptj.20110400

11

Harris, J. (2020). Key foundations of successful project planning and management. In J. Harris,

L. Roussel, C. Dearman, & P. Thomas (Eds.), Project planning and management: A

guide for nurses and interprofessional teams (3rd ed., pp. 1-14). Burlington, MA: Jones &

Bartlett.

Hoyer, E.H., Young, D.L., Klein, L.M., Kreif, J., Shumock, K., Hiser, S., Friedman, M.,

Lavezza, A., Jette, A., Chan, K., & Needham, D. (2018). Toward a common language for

measuring patient mobility in the hospital: Reliability and construct validity of

interprofessional mobility measures. Physical Therapy, 98(2), 133-142.

https://doi.org/10.1093/ptj/pzx110

Izumi S. (2012). Quality improvement in nursing: Administrative mandate or professional

responsibility? Nursing Forum, 47(4), 260–267. https://doi.org/10.1111/j.1744-

6198.2012.00283.x

Jones, R.A., Merkle, S., Ruvalcaba, L., Ashton, P., Bailey, C., & Lopez, M. (2020). Nurse-led

mobility program. Journal of Nursing Care Quality, 35(1), 20-26. doi: 10.1097/NCQ.

0000000000000404.

Liu, B., Moore, J.E., Almaawiy, U., Chan, W., Khan, S., Ewusie, J., Hamid, J.S., & Straus, S.E.

(2018). Outcomes of mobilization of vulnerable elders in Ontario (MOVE ON): A

multisite interrupted time series evaluation of an implementation intervention to increase

patient mobilization. Age & Ageing, 47(1), 112-119.

https://doi.org/10.1093/ageing/afx128

Pell Institute and Pathways to College Network. (2020). Education toolkit: Evaluation 101.

Retrieved from http://toolkit.pellinstitute.org/evaluation-101/evaluation-approaches-

types/

12

Teodoro, C.R., Breault, K., Garvey, C., Klick, C., O’Brien, J., Purdue, T., Stolaronek, A.,

Wilbur, H., & Matney, L. (2016). STEP-UP: Study of the effectiveness of a patient

ambulation protocol. MEDSURG Nursing, 25(2), 111-116. Retrieved from

http://web.a.ebscohost.com/ehost/pdfviewer/pdfviewer?vid=8&sid=98f2858a-e8aa-4dff-

b734-e9cf9dc3e831%40sdc-v-sessmgr02

Terhaar, M., & Sylvia, M. (2015). Scholarly work products of the doctor of nursing practice:

One approach to evaluating scholarship, rigour, impact and quality. Journal of Clinical

Nursing, 25, 163-174. doi: 10.111/jocn.13113

13

Appendix A

Data Collection and Evaluation Tools

Goal #1—Department Specific LOS

Department Month 1 LOS

Month