2020 vol. 44 no. 5 Australian and New Zealand Journal of Public Health 331 © 2020 The Authors

This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.

Despite the benefits of conducting evidence-based practice, many public health initiatives remain

unsupported by evidence1 and public health policies and practices that have been shown to improve health outcomes are not routinely implemented.2,3 Maximising the impact of public health interventions requires policy-makers and practitioners to use robust evidence to consider both ‘what’ interventions are effective in addressing public health issues and ‘how’ such interventions can best be implemented into practice. However, organisations that deliver public health initiatives face a range of barriers including a lack of skills and capacity when using and generating evidence to aid such decision-making.4,5 The development of decision support tools has been suggested as a useful strategy to help overcome such barriers.6

Decision trees are frequently used tools in health care to assist clinicians to make evidence-based diagnostic and therapeutic decisions.7 Such tools may also be useful for public health policy and service delivery organisations to aid their selection of evidence-based interventions and implementation strategies, and also to identify where further evidence needs to be generated. While a number of process models and decision trees for the medical and nursing field have been published,3,8 few of these address consideration of evidence- informed implementation strategies or evidence generation needs.

In this editorial, we describe a decision tree (Figure 1) developed and utilised in a large public health organisation in NSW, Australia.9

The decision tree tool aims to assist in the application of research evidence to maximise the impact of public health programs and services. The tool helps identify when there is sufficient evidence to support the delivery of particular services, and when there is not. The latter outcome provides an indication of where further research may be needed, identifying opportunities to undertake policy and practice relevant research. At each step in the decision tree, users are posed a question, and based on their response, a service and/or research action is suggested. Health services may have the capacity and expertise to conduct research actions or they may need to commission, partner or collaborate with researchers to do so. This tool should be used with other resources such as the Intervention Scalability and Assessment tool to determine intervention suitability for scaling up.10 Such assessments need to consider end-user values, resource, capability and context.

Step 1. Assessment of intervention options to address health problem

The Public Health Research and Practice Decision Tree starts at the point where the public health service organisation requires information regarding effective interventions to address an identified health problem. Systematic reviews are a recommended source of such evidence.11 Health services could employ or train staff, or engage a research organisation to critically appraise the findings of such reviews where they exist, or undertake a review where a contemporary review does not meet their needs. If a review identifies that effective interventions exist, the decision-maker moves to Step 2 of the tree. If the review identifies either: i) an absence of evidence regarding the impact (adverse or beneficial) of interventions on the health issue; ii) insufficient evidence; or iii) effective interventions that are not suitable for implementation in the local context, (e.g. cannot feasibly be delivered at scale), the conduct of further research is desirable to support intervention selection. Other frameworks have described factors that need to be considered when determining the suitability of an intervention for scaling up, including the severity of the problem it is seeking to address, the strategic/ political context, the intervention costs and benefits to the organisation, fidelity and adaptation to the original program, reach and acceptability, delivery setting and workforce, implementation infrastructure and sustainability.12 To meet this evidence need, public health service organisations

doi: 10.1111/1753-6405.13023

Improving the impact of public health service delivery and research: a decision tree to aid evidence-based public health practice and research Luke Wolfenden,1,2 Christopher M. Williams,1,2 Melanie Kingsland,1,2 Sze Lin Yoong,1,2 Nicole Nathan,1,2 Rachel Sutherland,1,2 John Wiggers1,2

1. School of Medicine and Public Health, The University of Newcastle, New South Wales

2. Hunter New England Population Health, New South Wales

Figure 1: Public Health Practice and Research Decision Tree.

Editorial

332 Australian and New Zealand Journal of Public Health 2020 vol. 44 no. 5 © 2020 The Authors

Editorial

could undertake research, partner with a research organisation, or commission such a trial to test the impact of a new or adapted intervention that aligns with the health service values, capability, infrastructure and context.

Step 2: Assessment of evidence- practice gaps

Once an effective intervention option has been identified or developed, an assessment of the extent to which it is currently being implemented in practice is required (evidence-practice gap assessment).13 Such assessments identify service delivery gaps that may benefit from investment in strategies to improve intervention implementation. Given the importance of equity for many service organisations, this assessment should address gaps in implementation across population sub-groups. Evidence-practice gap assessments can be conducted by service delivery staff or in partnership with researchers through an analysis of routinely collected administrative or service data, or by purpose-specific data collection activities including surveys, stakeholder engagement processes, or service delivery observations. If an evidence-gap assessment reveals effective interventions are being routinely implemented, and according to a sufficient standard across population sub- groups, no further investment in enhancing implementation is required. Nonetheless, a monitoring strategy is recommended to ensure implementation is maintained. Existing public health surveillance systems could be used for this purpose,14 or local monitoring or data collection systems could be developed.

Step 3: Assessment of implementation options

When an evidence-practice gap for a suitable and effective intervention is identified, the service organisation needs to identify effective strategies to ensure adequate implementation of the intervention. Again, systematic reviews can be used or undertaken to assess the effectiveness of implementation strategies. However, the effects of implementation strategies are likely to be contextually dependent, and so the selection of appropriate strategies should also be guided by local data on implementation barriers. Together with systematic review evidence, the use of theoretical frameworks

can help to select potentially effective strategies to overcome implementation barriers that have been identified locally.15 Effective and contextually relevant strategies that can be feasibly delivered within the resources and infrastructure available should be preferenced and employed to implement the intervention. Ongoing monitoring is also recommended to: i) ensure the implementation occurs as planned; ii) afford early identification and response to implementation or sustainability challenges; and iii) provide a mechanism for performance accountability.

If no effective and contextually appropriate implementation strategies are identified through this process, public health services may undertake, partner or commission an implementation trial to test the impact of an appropriate implementation strategy.

Conclusion

The decision tree is a simple resource intended to assist health service practice and to foster the conduct of practice relevant research. The tree has the potential to improve the impact of public health research by identifying opportunities where the enhanced alignment of research with the evidence needs of end-users is needed.

References 1. Indig D, Lee K, Grunseit A, et al. Pathways for scaling

up public health interventions. BMC Public Health. 2018;18:68.

2. Wolfenden L, Nathan N, Janssen LM, et al. Multi- strategic intervention to enhance implementation of healthy canteen policy: A randomised controlled trial. Implement Sci. 2017;12:6.

3. Hills A, Nathan N, Robinson K, et al. Improvement in primary school adherence to the NSW Healthy School Canteen Strategy in 2007 and 2010. Health Promot J Austr. 2015;26:89-92.

4. Oliver K, Innvar S, Lorenc T, et al. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14:2.

5. Tabak RG, Padek MM, Kerner JF, et al. Dissemination and implementation science training needs: Insights from practitioners and researchers. Am J Prev Med. 2017;52:S322-S9.

6. Yost J, Dobbins M, Traynor R, et al. Tools to support evidence-informed public health decision making. BMC Public Health. 2014;14:728.

7. Podgorelec V, Kokol P, Stiglic B, et al. Decision trees: An overview and their use in medicine. J Med Syst. 2002;26:445-63.

8. Titler MG, Kleiber C, Steelman VJ, et al. The Iowa model of evidence-based practice to promote quality care. Crit Care Nurs Clin North Am. 2001;13:497-509.

9. Wolfenden L, Yoong SL, Williams CM, et al. Embedding researchers in health service organizations improves research translation and health service performance: The Australian Hunter New England Population Health example. J Clin Epidemiol. 2017;85:3-11.

10. Milat A, Lee K, Conte K, Grunseit A, Wolfenden L, van Nassau F, et al. Intervention Scalability Assessment Tool: A decision support tool for health policy makers and implementers. Health Res Policy Syst. 2020;18(1):1.

11. Chambers D, Wilson PM, Thompson CA, et al. Maximizing the impact of systematic reviews in health care decision making: A systematic scoping review of knowledge-translation resources. Milbank Q. 2011;89:131-56.

12. Milat AJ, Bauman A, Redman S. Narrative review of models and success factors for scaling up public health interventions. Implement Sci. 2015;10(1):113.

13. Kitson A, Straus SE. The knowledge-to-action cycle: Identifying the gaps. Can Med Assoc J. 2010;182:e73.

14. Conte KP, Groen S, Loblay V, et al. Dynamics behind the scale up of evidence-based obesity prevention: Protocol for a multi-site case study of an electronic implementation monitoring system in health promotion practice. Implement Sci. 2017;12:146.

15. French SD, Green SE, O’Connor DA, et al. Developing theory-informed behaviour change interventions to implement evidence into practice: A systematic approach using the Theoretical Domains Framework. Implement Sci. 2012;7:38.

Correspondence to: A/Prof Luke Wolfenden, Hunter New England Population Health, Locked Bag 10, Wallsend, 2287 NSW; e-mail: [email protected]

© 2020. This work is published under http://creativecommons.org/licenses/by-nc-nd/4.0/(the “License”).

Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.

https://doi.org/10.1177/00333549211002767

Public Health Reports 2022, Vol. 137(3) 471 –478

© 2021, Association of Schools and Programs of Public Health

All rights reserved. Article reuse guidelines:

sagepub.com/journals-permissions DOI: 10.1177/00333549211002767

journals.sagepub.com/home/phr

Research

1 Cancer Prevention Fellowship Program, Division of Cancer Prevention, National Cancer Institute, Rockville, MD, USA

2 Herschel S. Horowitz Center for Health Literacy, School of Public Health, University of Maryland, College Park, MD, USA

Corresponding Author: Heather Platter, PhD, MS, National Cancer Institute, Division of Cancer Prevention, Cancer Prevention Fellowship Program, 9609 Medical Center Drive, Rockville, MD 20850, USA. Email: heather. platter@ nih. gov

The Value of Community Health Literacy Assessments: Health Literacy in Maryland

Heather Platter, PhD, MS1,2 ; Katya Kaplow, BS2 ; and Cynthia Baur, PhD2

Abstract

Objective: Community health assessments have typically not measured health literacy at the community level. We developed the Community Health Literacy Assessment (CHLA) framework to describe county and regional health literacy activities, assets, gaps, and opportunities in Maryland.

Methods: We implemented the CHLA framework in Maryland from January to August 2018. We conducted an environmental scan of Maryland’s 24 counties to identify community resources, health indicators, and organizations. We targeted local health im- provement coalitions and health departments for interviews in each county. We conducted qualitative interviews to understand what key community organizations throughout Maryland are doing to implement health literacy best practices and policies. We used summative content analysis to review, quantify, and interpret interview data.

Results: We conducted 57 interviews with participants from 56 organizations representing all 24 counties in Maryland. We cap- tured data on multiple dimensions of health literacy and identified 3 main themes: health literacy assets and activities, health literacy gaps, and health literacy opportunities. The most cited asset was collecting data to inform health literacy practices, the most cited gap was using jargon with community members, and the most cited opportunity was increasing public awareness of existing health programs through improved outreach and teaching health information–seeking behaviors.

Conclusion: A systematic community health literacy assessment is a feasible way to collect a large amount of health literacy data, which can inform strategic planning, determine community interventions, and ultimately lead us toward a health- literate society. We recommend that others replicate the CHLA framework to operationalize health literacy as a health indicator and include it as a community health assessment measure.

Keywords

health literacy, community health assessment, community health literacy assessment, systematic framework, public health

The Healthy People 2030 Secretary Advisory Committee states that health literacy occurs when a society provides accurate health information and services that people can easily find, understand, and use to inform their decisions and actions.1 This description shifts the focus of health literacy work from measur- ing and intervening to change people’s health literacy skills to measuring and changing professional, organization, and community- level practices and policies to address health liter- acy problems and barriers. The 2030 description aligns with rec- ommendations in the National Action Plan to Improve Health Literacy, which outlines ways to improve the nation’s health lit- eracy and create a health- literate society by having public and private sector organizations, community- based organizations, health care and education professionals, and policy makers work together to develop and use health literacy best practices.2 A

focus at the professional, organization, and community level suggests the need for assessment tools and data that identify activities and gaps in these contexts and indicate opportunities for improvement. Although some state- level health literacy data on people’s health literacy skills are available in the 2003 State

472 Public Health Reports 137(3)Public Health Reports 00(0)2

Assessment of Adult Literacy3 and, more recently, the 2016 Behavioral Risk Factor Surveillance System survey,4 these population- level data do not characterize what professionals, organizations, and communities are doing to address health liter- acy in their own geopolitical areas.5

Organizations can choose from several tools to assess their internal health literacy practices and policies, such as the Agency for Healthcare Research and Quality Health Literacy Universal Precautions Toolkit,6 the Ten Attributes of Health Literate Health Care Organizations,7 the HLE2 Assessment Tool,8 and the Health Literacy Champions Toolkit.9 However, the health literacy field lacks frameworks and assessment tools that extend beyond single organizations to facilitate descriptions, data col- lection, and analysis of multiple organizations, communities, or even geopolitical areas (ie, county) that often share public health responsibilities and resources that could affect health literacy.10 To address this gap, a team from the University of Maryland Horowitz Center for Health Literacy, which included the authors (H.P., K.K., C.B.), adapted the standard community health assessment process to create a Community Health Literacy Assessment (CHLA) framework. Maryland was an ideal state in which to test the framework because some organizations in the state have made health literacy a priority. Maryland has state laws encouraging health literacy education and training for stu- dents and practicing health professionals, and the Center for Health Literacy leads a statewide coalition. The CHLA frame- work, which is described elsewhere,11 allowed our team to cap- ture data on multiple dimensions of health literacy at the county and local coalition levels and across jurisdictions.

Previously, our team11 proposed that the community health assessment model used by public health departments and hospi- tals to gather important community health data in a geographi- cally defined area could be adapted for health literacy work.12

The framework has 7 steps that organizations can follow to col- lect health literacy information at the community, county, region, or state level (Figure). We found that the adapted framework gives local and county organizations the opportunity to charac- terize health literacy both internally and across organizations, and we described participant descriptions of health literacy and organizational rankings on the use of health literacy best prac- tices.11 This study reports the results of qualitative analyses of our Maryland assessment in more detail using the adapted framework.

Methods

We conducted an environmental scan to gather background data on Maryland’s 24 counties. The study team created a template for each county in Maryland to methodically record important information found through internet and database searches. The environmental scan included a review of publicly available community health assessment reports, county indicators, popu- lation demographic data, and local health organizations. The team examined available community health assessments to understand county health priorities and determine whether counties were implementing health literacy activities. After the environmental scan, the team developed an interview protocol, which included an email template for scheduling interviews and for talking with local health improvement coalition (LHIC) staff members to gain access to their membership list to schedule additional interviews with community leaders, and a semi- structured interview guide. We pilot- tested the semi- structured interview guide with 2 health department staff members to assess question flow, clarity, and understandability. After updat- ing the interview guide, the team conducted semi- structured

Figure. Community Health Literacy Assessment framework.

Platter et al 473Platter et al 3

interviews with community partners and leaders from community- based organizations, health departments, health care facilities, and LHICs throughout Maryland. We used a qualitative approach to understand what key community organi- zations, health care facilities, and health departments throughout Maryland are doing to implement health literacy best practices and policies to improve individual, community, and population health literacy.

Sample Our team used purposive sampling to identify and select partic- ipants who were able to provide information about the health literacy activities, assets, gaps, and opportunities within their organization, county, or region. We used information from the environmental scan of Maryland’s 24 counties (Baltimore City counted as a 24th county per state guidelines) to create a priority list for recruiting potential participants. We targeted LHICs first, because they include local leaders and community partners who determine and address public health priorities in their communi- ties. It was also valuable to connect with participants from local health departments because they often lead community health assessments in their county. We also contacted hospitals, public libraries, federally qualified health centers, faith- based organiza- tions, and nonprofit organizations, although not in any particular order. We used snowball sampling methods to identify addi- tional potential participants.

The University of Maryland Institutional Review Board determined this project to be exempt before data collection. We contacted participants and asked them to participate in a semi- structured in- person or telephone interview. Participants gave verbal consent to participate and allow the research team to tran- scribe detailed notes during the meeting. Interviews ranged in length from 30 to 60 minutes, and no compensation was offered. We used the tested interview guide to facilitate each interview. Trained team members who conducted the interviews updated and cleaned the interview notes, which were similar to a tran- script, and then uploaded and stored them in password- protected files. Additional details about the methods can be found elsewhere.11

Data Analysis We used summative content analysis to review, quantify, and interpret interview data.13 We individually reviewed and com- pared interviews to create codes for health literacy activities, assets, gaps, and opportunities. A code is a label that represents a group of similar interview statements or phrases. For example, we coded a discussion about gaining community input for health material development through the use of evaluation surveys as “asset of community input.” We quantified codes by the number of participants who mentioned a phrase in an interview that matched the code. We counted interviews with organizations that serve multiple counties, such as a tri- county LHIC, for each county served by the organization. The unit of analysis for this

study was the county, providing a denominator of 24. Codes were open to revision and updated after every few interviews because the coding process took place while interviews were still occurring. Interviews concluded when saturation was reached for each county or no other participants for the county were available for interviews.

To assess reliability, the first author (H.P.) met biweekly with the second author (K.K.) to create and review codes. After the codebook was created and interviews were completed, 2 inde- pendent coders (H.P., K.K.) coded 6 of the same interviews and compared codes, which were consistent. To further enhance reli- ability, member checking was performed with participants (n = 10) and other community members who attended a community results forum (n = 30), to determine their agreement with inter- preted results.

Results

We completed 57 interviews with participants from 56 organiza- tions representing all 24 counties in Maryland during a period of 7 months. Eighteen counties were represented by interviews with a leader from their county LHIC, and a minimum of 2 inter- views were completed with organizations per county or LHIC, ensuring that all 24 counties in Maryland are represented in the sample and results. Multiple types of organizations were included in the study (Table 1). Three themes arose during our analysis of the interviews: health literacy assets and activities, health literacy gaps, and health literacy opportunities. We com- bined activities and assets into 1 theme because we considered health literacy activities to be assets.

Health Literacy Assets and Activities Participants mentioned 5 key health literacy–oriented activities or assets currently implemented in their communities (Table 2). Participants representing all 24 counties in Maryland mentioned collecting data to inform health literacy practices and discussed

Table 1. Types of organizations participating in interviews about health literacy best practices and policies, Maryland, 2018

Organization type No. of counties

(N = 24)

No. of participants

(N = 57)

Local health improvement coalition

18 21

Health department 7 12

Hospital 6 7

Community health center 5 6

Literacy council, public school, and public library

5 5

Nonprofit organization 3 3

Faith- based organization 2 2

Cooperative extension 1 1

474 Public Health Reports 137(3)Public Health Reports 00(0)4

community outreach and educational materials, such as sharing health education print or digital materials to improve patients’ ability to manage their own health.

Another common activity and asset mentioned by partici- pants representing 23 counties was community outreach and education materials, where organizations provide in- person health education to community members in a way that they understand. Navigation services was another asset described by participants representing 21 counties and defined as an activity to improve a person’s ability and efficacy to take control of their own health. Participants representing 21 counties mentioned evaluating their materials for health literacy, although most par- ticipants did not specify a tool or technique for evaluation. Only 1 participant mentioned using the CDC Clear Communication Index14 to evaluate materials.

Health Literacy Gaps Participants identified 5 health literacy gaps (Table 3). Jargon was a prevalent gap mentioned by participants representing 19 counties. Participants defined jargon as words used by medical providers, programs, or health care workers that may be difficult to understand, and no explanation of these words is provided in plain language to community members. One participant repre- senting several counties said that “[e]veryone is using acronyms, [which is] so confusing.”

Another gap, mentioned by participants from 17 counties, was limited funds dedicated to health literacy activities. One participant from a rural county shared that there are “no separate funds related to health literacy incorporated into the budget.” Seventeen participants also noted that no staff members were dedicated to conducting health literacy–related work in the community.

Participants from 13 counties stated that they had no process in place to evaluate the effectiveness of their programs or mate- rials in relation to health literacy. Participants from 8 counties mentioned a lack of advocacy for the importance of health liter- acy and the improvement of programs through implementing health literacy techniques and best practices.

Health Literacy Opportunities Participants identified 5 opportunities to improve health lit- eracy within their organization, community, or county (Table 4). Participants representing 21 counties discussed the need to increase public awareness of existing health pro- grams and services through improved outreach and by teach- ing better health information–seeking behaviors. Participants from organizations in 19 counties mentioned the need to bring health literacy into discussions at organizational meet- ings to raise awareness of the importance of health literacy in all aspects of the health care system. Participants

Table 2. Health literacy assets and activities reported by interviewees (N = 56), Maryland, 2018a

Assets/activities No. of counties

(N = 24) Quotes

Data collection to inform health literacy practices

24 Through the needs assessment, we know education and literacy levels are lower than the state average. We did do a survey of clinics and health departments 5 years ago to ask about health literacy and found that patients don’t get help filling out forms or understand[ing] prescriptions.

—Local health improvement coalition

Community outreach and