Dose Matters: An Approach to Strengthening Community Health Strategies to Achieve Greater Impact
Abstract
Dose Matters describes the concept of “population dose”—an approach to strengthening and evaluating the impact of complex multisector, multilevel, place-based initiatives. This discussion paper reports on what is promising about the approach while recognizing the measurement and other challenges that still lie ahead. The concept emerged from ongoing evaluations of Kaiser Permanente’s Community Health Initiative investments, conducted by the Center for Community Health and Evaluation (CCHE) at the Group Health Research Institute in collaboration with partners at the University of California.
Following an overview of the Community Health Initiative, this report describes the concept in more detail and explains how it can be used to estimate the population-level impact of a set of related interventions. The estimated impact or dose of an intervention is the product of reach (the number of people touched by the intervention) and strength (the effect size or impact on each person reached). The dose concept can also be used in planning and implementation to yield greater impact.
We hope that Dose Matters will introduce the concept of population dose in enough detail that a broad audience of community health researchers, evaluators, practitioners, and planners will be both prepared and eager to apply these analyses and approaches to their own work. Additional guidance is available in the form of an interactive toolkit developed by CCHE and Kaiser Permanente (see the Related Resources box). Together, Dose Matters and the toolkit will describe in detail the uses of dose for both evaluators and researchers, who will use the quantitative calculations, and community-based organizations and funders, who will apply the overall concept to strengthen every phase of improving community health outcomes, from planning to implementation to evaluation.
Introduction
In a neighborhood in one of Kaiser Permanente’s regions across the country, children walk or ride their bikes to school, taking advantage of a Safe Routes to School initiative that has helped get kids out of cars and buses and onto sidewalks.
At school, cafeterias and vending machines offer healthier fare – less sugar and fat and more grains, fruits, and vegetables. A district-wide curriculum puts the “physical” back in physical education (PE), adding minutes of activity to each child’s daily routine.
At work, the children’s parents are taking the stairs and getting a nudge to participate in worksite wellness programs. Around the neighborhood, more farmers’ markets are operating year-round. Open spaces, safer trails (Figure 1) and parks invite people of all ages to resist the lure of their televisions or computer screens and gather, walk, and play instead.
These types of changes have been gaining traction and support for more than a decade, emerging as responses to the obesity epidemic and its toll on the health, economic futures, and well-being of so many Americans. The quest has been not only to improve individual health behaviors by encouraging people to transform unhealthy lifestyles into healthier eating and more active living, but to do so on a scale and significance that yields outcomes at a population level.
Doing so requires much more than individual coaching, support, and the fits and starts of attempts to change behavior—the kinds of interventions that might happen through a doctor’s office and help some, but far too few, among those who could potentially benefit. We can achieve lasting behavior change on a larger population scale when healthier choices replace and outnumber unhealthy choices wherever we live, work, learn, pray, and play. That means influencing the policies and environments that surround us and doing so in many dimensions and layers simultaneously.
These multi-dimensional, community-wide health initiatives go by many names and labels: place-based, multisector (e.g., schools, worksites, and neighborhoods), multilevel (e.g., individuals, families, and communities or local, state, and federal), and integrated (across several of these dimensions). Within Kaiser Permanente, as described below, they are at the core of a decade-long Community Health Initiative (CHI) focused on increasing opportunities for healthy eating and safe, active living.
Asking and Answering “What Works?”
Regardless of their scope, nomenclature, or funding source, these multidimensional, community-wide health initiatives are complex: they are complex to implement and even more complex to evaluate. As more planners, researchers, funders, practitioners, and advocates gain experience with these initiatives, the question of “What works?” looms large. Where and how should we invest our scarce community health resources—including not only funding but also the time, energy, and collaborative capital of the many agencies and individuals involved?
Unfortunately, the existing evidence base, although improved over the past decade, still offers limited guidance about what works and under which conditions. In part, this is because the main body of research so far has focused on health promotion and programs rather than on broader environmental and policy interventions. Even when the existing evidence base offers guidance or insights about a particular program or policy, it is typically not generalizable to other locales and portfolios of interventions, limiting its usefulness. The evidence base also offers few clues about the relative impacts of different strategies (Cheadle et al., 2013). Will greater behavioral change be achieved by building new sidewalks, or would it make better sense to improve public parks or focus on improved PE – or other portfolios of interventions? What are the best choices for a community from the possible menu of interventions – or which combinations offer even greater impact, savings, and/or improved health outcomes?
About This Report
Dose Matters describes the concept of “population dose” – a way to overcome some of the challenges inherent in creating and evaluating complex multisector, multilevel, place-based initiatives. As described below, the population dose concept and its application are still a work in progress; Dose Matters reports on what seems promising about the approach while recognizing the measurement and other challenges that still lie ahead. The concept emerged from ongoing evaluations of Kaiser Permanente’s CHI investments, conducted by the Group Health Cooperative’s Center for Community Health and Evaluation in collaboration with partners at the University of California.
Following an overview of the CHI and how it led to the formulation of population dose, this report describes the concept in more detail, explains how it can be used to measure an intervention’s (or set of interventions’) reach and strength, and reviews the implications for strengthening dose in order to yield greater impact.
We hope that Dose Matters will introduce the concept of population dose in enough detail that a broad audience of community health researchers, evaluators, practitioners, and planners will be both prepared and eager to apply these analyses and approaches to their own work. As this occurs, we look forward to collecting even more examples of how dose has been applied in communities and regions across the country, adding to the collective knowledge base about estimated effects and thus making the concept even more useful and applicable over time.
Additional guidance is available in the form of an interactive toolkit developed by the Center for Community Health and Evaluation and Kaiser Permanente. Together, Dose Matters and the toolkit should help address questions about the population dose concept and how to operationalize it at several different levels. The quantitative calculations and methods involved in estimating dose can be tricky and may be of greater interest to evaluators and researchers. The evolving quantitative methods are briefly reviewed in this report, but the focus is on the overall concept and how it can be applied to strengthen every phase of improving community health outcomes, from planning to implementation to evaluation.
The Kaiser Permanente / Community Health Initiative Context
Kaiser Permanente, founded in 1945, is America’s oldest and largest private nonprofit health care organization. Today, Kaiser Permanente’s 240,125 employees and physicians provide health care through 35 medical centers and 431 clinics in nine states and Washington, DC.
Kaiser Permanente’s clinical role has always been embedded in broader notions of community health, recognizing that access to medical care, while important, pales in significance when compared to the contributions of behavioral, environmental, and social factors that influence health and wellness. For the past decade, building on a history of similar initiatives, Kaiser Permanente has augmented and expanded its clinical health care role with a focused and ambitious community-based obesity prevention effort: the Community Health Initiative (CHI). CHI is a prevention-driven approach to supporting policies and environmental changes that increase access to nutritious foods, physical activity, economic vitality, safety, and wellness in local schools, workplaces, and neighborhoods. As a multisector, place-based effort, CHI aligns with Kaiser Permanente’s broader Total Health strategy – an effort to deploy Kaiser Permanente’s health care system, workforce, and business operations in tandem to create health for its members, workforce, and the broader community.
Figure 2 shows CHI’s presence across Kaiser Permanente regions, reaching more than 1.1 million people since CHI was launched in 2004. The most substantial and mature initiatives are in Colorado, with 32 CHI sites.
The $50 million invested by CHI to date crosses settings – health care, neighborhoods, schools, and worksites – and supports a variety of strategies, often in combination. These strategies include, for example, environmental changes such as boosting the healthy choices offered in school cafeterias; programs such as worksite wellness offerings; policies such as requiring PE across a school district; and capacity building (e.g., leadership or partnership development, advocacy training, and skills in specific relevant topics such as interpreting data, planning, implementation, and evaluation) that makes these strategies more effective and feasible.
When does an activity become a strategy? For CHI’s purposes, a strategy is any set of coordinated activities that are designed to lead to a sustainable community change. Each part of this definition is important. The “set of coordinated activities” means the activities have some intention and coordination behind them, even though unplanned events arise that influence these efforts. A “sustainable community change” is a change in community programs, policies, or the environment that supports improved food and physical activity behaviors. These, in turn, could be grouped according to the behavior change being sought (e.g., nutrition or physical activity) or by their potential scope or impact (e.g., hallway signs to encourage taking the stairs versus worksite wellness programs that promote walking clubs at lunchtime).
Figure 3 shows the types of strategies implemented in 35 of the 50 CHI communities for which detailed data were available in early 2014 – then constituting at least 600 different strategies.
CHI Evaluation and Strategy Refresh: Origins of the Dose Concept
In 2009, with hundreds of different strategies and combinations of strategies deployed in settings across the country over a 5-year period, it was time to ask: Is CHI making a difference in people’s health? Are these strategies, singly or in combination, strong enough to have an effect?
The CHI evaluation was framed by a cross-site logic model, provided in Appendix A. The evaluation focused on intermediate outcomes of how health-promoting system changes affect increases in healthy eating (nutrition) and active living (physical activity). In addition, the evaluation explored the degree to which changes in community capacity occur and support these changes.
With the cross-site logic model as a guide, Kaiser Permanente’s CHI team and evaluators designed an ongoing, multi-method evaluation framed by these specific questions:
- Are community food and activity environments changing?
- Are we having an impact on behavior, health status, and community capacity?
- What are we learning to inform program improvement?
The team sought answers to these questions using a combination of methods—key informant interviews, documentation of intervention progress, surveys assessing behavior changes at both the individual and population levels, and Kaiser Permanente member data (e.g., health outcomes such as body mass index.)
The preliminary results led to some midcourse adjustments that evolved into the concept of dose, influenced in part by a framework originally set forth by Glasgow et al. (2006) as REAIM (Reach, Effectiveness, Adoption, Implementation, Maintenance).
The early CHI assessments suggested that in many cases CHI strategies were not strong enough to have the expected impact. Community food and activity environments were changing, and CHI was having some impact on behaviors, health status, and community capacity. But were CHI strategies consistently strong enough to reach enough people and eventually change people’s health throughout a community? Unfortunately, they were not as strong as we had hoped, expected, and would like to see, given the investments of CHI’s first 5 years.
Why not? In exploring the differences between CHI strategies that appeared to yield results and those that did not, CHI and its evaluators learned that many interventions were weak, with resources spread too thinly across sites to make a real difference.
What did this mean for the next 5 years and beyond? We realized that we need to touch more people in a given place and that that touch must be strong enough to have a realistic, expected chance to make a difference in behavior – to have an impact. This idea – that there is an amount of “dose” underlying measurable changes and that it is a function of both the number of people touched as well as the strength of that touch – became the concept of “population dose.”
This insight about what might explain differences in CHI outcomes affected our ongoing evaluation in several ways. First, it led to changes in our expectations about specific interventions – for Kaiser Permanente as a funder and for our grantees as well. We also developed analytic methods, technical assistance, and evaluation systems that take dose into account, as described in greater detail below. Finally, we recognized that community ownership is important for increasing the chances of sustainable change and long-term impact.
The concept of dose was useful in looking back and understanding what had occurred (and not occurred) as a result of CHI so far. But how do we know how strong a strategy or combination of strategies needs to be in the future to achieve an impact in a given population? How can we prioritize interventions to have the greatest impact?
Answering these questions meant exploring the role of evaluation in the planning and implementation phases of working with CHI communities to build stronger interventions and accelerate our work together. We also had a significant challenge: finding a way to quantify dose so we could capture the complementary nature of multiple intervention “apples and oranges” with a common metric and then use that metric to estimate population health improvement.
In the sections that follow, we explain the concept of population dose in terms of its components (reach and strength) and then explore some potential ways to calculate dose, which is very much a work in progress. These sections are followed by examples of one of the most important implications of dose: how communities can use the concept to build stronger and more effective interventions, no matter where their starting point may be.
Population Dose: The Concept
The concept of population dose gives us a way to describe and compare the relative impact of different strategies. A strategy is considered high dose if its implementation results in many people in the community changing their lifestyles in significant ways. These components work together to create population dose, which is an estimate of the impact of community interventions on an average resident’s behavior.
Many People = Reach
The “many people” part can be thought of as reach: the number of people from a particular target population touched by or exposed to an intervention strategy, expressed as a percentage of the target community population. For an example of calculating reach, see Box 1.
To understand the number of people touched by an intervention, we need to be clear about who is and is not included in a target population. In other words, the denominator is key. Defining and understanding the denominator or complete target population gives us options for comparing interventions and for understanding what an “average” resident might experience in terms of exposure to an intervention, because some people will receive or be exposed to an intervention, and some will not.
For example, consider a specific community – whether it is a town, school district, census tract, county, or some other geographically defined place. Now consider all the people living in this community to compare two interventions: an exercise class and a new sidewalk.
Even if the exercise class “reaches” a lot of people, it would still typically be considered a low-reach intervention because the number of people it reaches is expressed as a percentage of the total community population (or, in some cases involving children and youth, a school population). As a result, even if dozens or hundreds of people attend the exercise class and benefit from it, the reach into the total population is low.
The new sidewalk, on the other hand, could reach many more people and a significant proportion of the community, depending on its location and use patterns. Nevertheless, it might not have a huge impact on each person. In describing and measuring dose, we often have to say, “It depends” for this reason: context, including implementation, is crucial.
Changing Their Lifestyles in Significant Ways = Strength
As in the examples above, we can calculate an intervention’s reach—the number of lives touched by or exposed to an intervention, expressed as a percentage of the target community population (the denominator).
What about the intervention itself? We want to know the degree to which people changed their health behavior choices as a result of being exposed to the intervention, ideally on a daily basis. Were they physically active for more minutes, every day or most days, compared to before? Did they eat more fruits and vegetables, not just once in a while but on a daily basis, compared to where they started before the intervention? Did they gradually decrease their daily intake of sugar-sweetened sodas?
Ideally, evaluations and published research help us express strength in terms of effect size—the average percent change in behavior for each person exposed. For example, a researcher could have conducted a study of a new school PE curriculum, finding that students in the PE class increased their minutes of moderate/vigorous physical activity by 20 percent. If another community implemented a similar program, a 20 percent effect size would be reasonable to assume if the implementation and population were similar.
An effect size could also be calculated for a particular intervention or program. For example, if participants in a produce cooking class ate four servings each day of fruits and vegetables before taking the class and five servings after taking the class, the effect size would be 25 percent (the one-serving difference between four servings before and five servings after the class, divided by the baseline of four servings before the class).
In reality, we know that strength can be influenced by the following:
- the frequency of someone’s exposure (all or most days or just once in a while?);
- the intensity of exposure (as above, the exercise or PE class versus a place where one could walk a bit more if sufficiently motivated and convenient); and
- the degree to which a healthy choice is really the choice (for example, all healthy choices in the vending machine or some healthy choices mixed with less healthy and perhaps more tempting ones).
In the many cases in which we may not have a rigorously researched, published effect size to work with, we rely on estimates about each of these factors—frequency, intensity, and the degree to which a healthy choice is a truly default choice—to capture the strength of a given strategy.
Calculating Population Dose: A Work in Progress
When we combine the number of people exposed (reach) and the estimated impact or change in lifestyle for each person reached (strength), we get population dose (Figure 4).
For more on calculating effect sizes, strength, and dose, see Appendixes B–E. CHI’s evaluators, the Center for Community Health and Evaluation and the University of California, have developed some preliminary effect size groupings on the basis of literature reviews and the CHI evaluation so far. These, along with a basic strength rating for strategies in physical activity, nutrition, or both categories, can be found in Appendixes B and C and are an extremely helpful starting point.
Appendix D explains how strength has been calculated for the main behavior changes of interest in CHI: physical activity, fruit and vegetable consumption, and sugar-sweetened beverage intake. Appendix E provides more detailed examples about how dose is calculated for different interventions.
We can place strategies according to whether they are high to low reach and high to low strength, creating four simple categories similar to the ones in Figure 5. To achieve population health changes, communities would aim to achieve both high reach and high strength (i.e., the upper-right quadrant on these two-by-two grids).
Sometimes low-reach or low-strength strategies are still worthwhile, especially if they do not displace or distract from higher-reach, higher-strength options. For example, lower-reach, lower-strength strategies may help build capacity or increase community buy-in and set the stage for later successes. They may also reach a particularly high-need subpopulation.
Also, note that most strategies can be designed and/or implemented to be high or low dose. In other words, an intervention could fit into different quadrants in the two-by-two grids, depending on how it is implemented. For example, a walk-to-school event could be organized once a year, once a semester, or more frequently—even every school day. It could reach a few students rarely, more students more frequently, or all students all the time. The more students it reaches, with more strength, the higher the dose.
When we apply calculations to the concept of population dose (see Appendixes B–E for more details on how this can be done), we see this same idea expressed quantitatively. For example, an intensive school-based physical activity program could reach just 5 percent of students but deliver a high strength rating (e.g., 10 percent). Its dose (reach x strength): 5 percent x 10 percent = 0.5 percent.
Likewise, 25 percent of people in a target community may live near a new walking trail, but the strength rating is low because only a few people use the trail each day (for estimating purposes, approximately 2 percent). The walking trail’s dose is thus 25 percent x 2 percent = 0.5 percent—exactly the same as the intensive school-based program.
Calculating effect sizes and measuring dose represent an ongoing work in progress currently being tested and refined by CHI’s evaluators. We recognize that we are in the piloting stage of measuring strength, effect size, and thus dose itself. Although these calculations and measurements are preliminary, they support the hypothesis that strong interventions contribute to population health and that it is possible to build and combine strategies in ways that increase the chance they will have an impact in a particular community. One important implication is that even when specific calculations and measurements are not available, the concept of population dose can still be used to plan and implement stronger, more effective strategies and interventions—as described in more detail in the next section.
Calculating Dose for Combinations or Clusters of Interventions
The examples in Appendix E show how dose can be calculated for a specific intervention. In practice, though, many multisector, place-based initiatives (including CHI) combine multiple strategies that collectively target the same outcome. How does dose apply in these situations?
For the CHI evaluation, strategies were grouped into dose clusters according to the main outcomes of interest: physical activity, fruit and vegetable consumption, healthy foods, and sugar-sweetened beverages. When strategies are clustered in this way, we do not assume that the whole is greater than the sum of the parts. Table 1 shows how dose was calculated for a cluster of five school-based strategies targeting physical activity in a CHI community.
In this particular example, a pre/post survey of students about their overall minutes of physical activity was expected to yield measurable change because the cluster dose was greater than 5 percent. Indeed, although some students would have shown higher changes in their physical activity minutes and some lower, the average did turn out to be measurable and significant according to the surveys, at 4.1 percent. This finding gives us confidence that results on the ground are more likely to come from the dose cluster created by strong CHI interventions.
(For more on the variations in estimated dose and how we are interpreting them, see below and Appendix B.)
Ranges and Types of Expected Impact
When we add the estimated doses for strategies in a cluster (from evaluations and the literature, as constructed in the examples above), we are trying to estimate whether the expected impacts—even small ones—can produce effects when they occur on a population-wide basis. For example, if a 2 percent average increase in physical activity minutes results in a 1 percent decline in obesity rates each year and a 0.5 percent long-term decline in the prevalence of diabetes, that can have an impact in a community of 10,000 people. Each year, following these estimates, 100 fewer people would be categorized as obese, and 50 fewer people would be diagnosed with diabetes (Homer et al., 2010).
Because small doses translate into significant changes as they affect entire populations (or portions of populations), we are able to categorize dose estimates as follows:
- A dose less than 2 percent is considered low impact; we would not expect a significant impact;
- A dose between 2 percent and 5 percent could yield a significant impact but may not be large enough to measure with a population-level survey (which has limited ability to detect small changes); and
- A dose above 5 percent would be both significant and measurable, detectable through a population survey.
At this point in our work, these thresholds or tiers are somewhat arbitrary. Even as rough categories, though, they can offer guidance: in general, higher population dose is more likely to yield measurable, detectable changes. Low dose strategies could be combined as part of a stronger dose cluster and/or strengthened in their own right to achieve population health outcomes. And even when estimated dose is relatively low and thus hard to detect through standard surveillance and surveys, it may still be worth considering.
In considering and comparing dose, note that the terminology of “expected” or “estimated” impacts is important. Before strategies have been implemented, we make an informed estimate or prediction of impact on the basis of calculations such as the ones above of reach and what we know or assume about strength and effect size from other similar interventions. (See Appendix E for more details about the specific components of these calculations.) Next, we measure actual reach through implementation data and obtain actual strength, to the extent possible, through strategy-level evaluations.
We measure impact in CHI communities through a population health survey, understanding that these types of data collection are not always possible, are often expensive and are unlikely to detect smaller effects. We also compare these results to secular trends from other data sources, such as the Behavioral Risk Factor Surveillance System or the Youth Risk Behavior Surveillance System over the same time period. If our hypothesis is correct that stronger-dose interventions can move population health, we expect to see measurable change, especially for clusters of strategies implemented together. The steps necessary to calculate population dose are highlighted in Box 2.
As we obtain more data on more communities and more clusters of strategies, we should have a greater spread of dose estimates and associated population-level changes. As a result, our analysis approach will shift to what is shown in Figure 6—correlating the dose on the x-axis to the population-level change on the y-axis. If the two align, we can be more confident that observed population changes are a result of the CHI strategies being implemented—strategies with a higher population dose generated by a combination of higher reach and/or strength. The measurement challenges are considerable, but as we learn more, these estimates can continue to be refined.
Early Findings
The 2009 evaluation of CHI offered early clues that dose matters. The next step was to confirm this with community-level evidence.
An initial review of survey data about population health changes in CHI communities did not yield differences with comparison groups. However, examining population-level results in communities with high-dose interventions told a different story. In almost half of those cases, results were significant—especially for youth interventions in schools targeting physical activity (Cheadle et al., 2012). As the examples below illustrate, we found that schools and school systems make high reach much easier to achieve than is the case in a broadly dispersed community (Figure 7).
- In 13 schools within a suburban Colorado community that participated in the Safe Routes to School initiative and implemented combinations of safety enhancements, infrastructure changes, and walk-to-school days, the percentage of children walking to school increased from 24 percent to 36 percent over a 2-year period.
- In Northern California, a combination of strategies—strengthening PE standards, adding physical activity breaks to after-school programs, implementing Safe Routes to School, including a walking school bus, and carrying out a community physical activity media campaign and school-based awareness strategies—led to an increase from 61 percent to 67 percent of children doing at least 20 minutes of vigorous physical activity per day.
- In a rural Colorado elementary school, a high-dose cluster of fruit and vegetable strategies included installing a salad bar and making other cafeteria changes, along with promotion. Evaluators measured an increase in self-reported fruit and vegetable consumption of 13 percent—perhaps even more significant among a target audience of finicky elementary school lunchers.
- Another Colorado community documented a significant, population-level increase of 4 percent in minutes per day of physical activity among youth after 3 years of high-dose, physical activity–focused strategies, which included a media/promotion campaign, implementation of action-based learning initiatives, and walk/bike to school and after-school physical activity programs.
For each of these CHI success stories in which clusters of strategies rated as “high dose” were associated with documented changes in behaviors, we also have stories of communities whose clusters of strategies were rated as low or medium, and we were unable to measure any change in behavior in these communities. Again, this does not necessarily mean that the strategies were not worth pursuing or did not achieve health outcome goals but that their effects could not be detected.
Still, these findings suggest that dose does indeed matter—and that without attention to achieving higher dose, we are unlikely to see health improvements at a population level. Moreover, by focusing on populations as the denominator for reach calculations, the CHI results have helped demonstrate that high doses and expected impacts can be achieved, even at relatively modest levels of per-capita changes. The 2 percent average increases in physical activity described above may not seem significant as a percentage, but across 10,000 people, they certainly are—especially for the 100 people not categorized as obese and the 50 not diagnosed with diabetes.
Practical Implications: Ways to Increase Population Dose
If dose matters and if high-dose strategies matter even more, how can we systematically and intentionally increase dose? How can we build dose into planning, technical assistance, strategy selection, and evaluation?
In part, we do so by asking the following: “Can we reach more people, more times, more often, for a longer period, throughout the year, and with supporting strategies across sectors?” The parts of this question are highlighted in Box 3. For almost any community health effort, this is an ongoing conversation. Even before engaging in quantitative calculations or rigorous evaluations, many members of CHI community collaboratives reported that they found dose useful at this level. For example, some CHI coalitions used the concept of dose during planning stages to identify evidence-informed strategies with a higher likelihood of having an impact on behavior and health outcomes than what they were doing before. An important corollary is to use the concept of dose to avoid or stop strategies that are failing to gain momentum so that those resources can be deployed elsewhere more effectively.
Collaboratives also used the concept of dose to communicate more clearly with residents and elected officials about why they were focusing on particular high-dose strategies and to build consensus and support for selected strategies. As one CHI coordinator explained, the concept of dose helped move the coalition beyond a desire to offer one-on-one classes. “We say to them: ‘If you do that class, what is the reach? How often are they getting it?’ It became a useful tool in working with the community.”
While many CHI communities expressed the wish that they had started with the concept of dose earlier in their intervention process, dose can be a helpful lens at any point: during initial or strategic planning, ongoing planning and check-ins, implementation, or evaluation phases of community health interventions.
The first step is to start wherever a setting, sector, or community might be. For example, a community may want to work on healthy vending machine policies by moving to half healthy options and then to all healthy options. If that turns out to be impossible, it may be preferable to remove vending machines altogether. Likewise, in a school setting, a group might begin by taking on the cafeteria’s menu offerings and then moving on to a la carte foods, competitive offerings, concessions, or options offered at sporting events and school celebrations—indeed, across the entire campus until all teachers, staff, and students are surrounded by healthy options every day. Once this is accomplished, they could move on to the food options near the school in corner stores or mobile vendors. With persistence and small wins, any point along this spectrum could lead to a much healthier environment inside and outside the school’s walls, when the healthy choice truly becomes the default choice. This process is illustrated in Figure 8.
A similar continuum applies to active living (Figure 9). Some communities may start with a walking school bus, gradually expanding it to more days, more children, and more schools until an entire district is covered, building both reach and strength along the way. As this occurs, other changes will be needed to make walking safer (for example, by adding crosswalks and sidewalks), which in turn will require engaging parents and reaching even further into the community. Inside a school, changes might involve revamping a PE curriculum, adding recess in classrooms, and filtering to after-school programs. Depending on the community, building sidewalks might be more feasible in one place than changing school PE policies, and vice versa.
In addition to the main intervention(s) it is planning to implement, a community might also consider promotional or educational strategies that support or nudge people toward a particular behavior that might not be high dose on its own but might become so when combined with other interventions. For example, a booth at a farmers’ market that teaches people how to prepare the fresh fruits and vegetables available there might not stand alone as a high dose intervention, it but would support other interventions—such as making farmers’ market produce more affordable for low-income families.
Increasing Dose by Increasing Reach
Increasing reach requires spreading changes to more people—more neighborhoods, more facilities, more schools—to increase the proportion of people within a community who are exposed to a strategy or cluster of strategies.
Increasing Dose by Increasing Strength
Increasing strength means increasing the potency of the intervention or strategy on those reached. For example, a starting point might be to replace half the items in a vending machine with healthy items. A stronger strategy would be to offer only healthy items (100 percent instead of 50 percent) or, better yet, to remove the vending machines altogether. If this occurred in one school, it could be extended to all schools in a district or to schools as well as corner stores.
Similarly, a “Walk to School” day is a good start. But wouldn’t a “Walk to School” year be even better? And what about a Complete Streets program that makes walking easier and safer for everyone, not just students? Complete Streets, coupled with a stronger school physical activity curriculum—inside and outside of PE class—would do even more. In school, physical activity breaks that go from 5 to 10 to 15 minutes per day add strength with every minute.
Increasing Dose in Specific Settings
As noted above, any setting—a neighborhood, a school, a workplace, or a public venue—can strengthen existing strategies no matter where these started. A cafeteria could meet nutrition guidelines and then remove unhealthy options and replace them with healthier ones, using pricing and placement options to encourage healthier choices and nutrition labeling to inform customers. Farmers’ markets can add nutrition or cooking education or offer Women, Infants and Children or Electronic Benefit Transfer options for low-income shoppers. Workplaces can allot more time for employee physical activity breaks and support active transportation through bike sharing, bike racks, on-site showers, and changing rooms.
Implications for Evaluation, Planning, and Technical Assistance
As noted above, one of the most powerful ways to use the population dose approach is in the earliest phases of planning—when strategies are still being designed and packaged and before they are implemented. Asking hard questions about dose—a strategy’s (or cluster of strategies) reach and strength, how each could be increased and how strategies could be combined to boost dose—could help agencies and coalitions select strategies that have a higher likelihood of achieving the health outcomes to which we all aspire. As we learned through our CHI midcourse corrections, asking these questions may be revealing in difficult ways, such as seeing interventions that are weaker than others or learning that dedicated efforts are too diffuse to make a difference. One outcome may be dropping some strategies in favor of others.
In some cases, these discussions will be internal. In others, some technical assistance from evaluators or other facilitators may be required to assess strategies from this vantage point and design the intermediate and outcome evaluations that help us understand whether our estimates of dose and impact are accurate.
As we continue to develop estimates of population dose for CHI strategies and gauge their impact using population surveys, we recognize the ongoing challenges. These types of evaluations are resource and time intensive; ideally, existing population health data can help supplement data for comparisons of estimated and measurable changes.
Even when strategies are effective, though, they may not yield significant and measurable findings. For this reason, measuring intermediate outcomes becomes vital for understanding and communicating impact (while keeping an eye on population health changes). Estimates that allow comparisons across strategies are particularly important for program design and evaluation, as well as for quality improvement purposes.
Conclusion
Exploring dose through the experiences of CHI communities across the country has taught us that interventions and strategies can be designed and implemented in specific ways—by increasing their reach and strength—to increase the likelihood of seeing health behavior improvements at a population level. The population dose concept has also given us an array of methodological tools that allow us to add and compare apples and oranges (to use a fresh produce analogy) and estimate impacts going forward.
The measurement aspects of dose are still a work in progress, but the CHI team at Kaiser Permanente and our evaluation colleagues at the Center for Community Health and Evaluation and the University of California are encouraged by the many ways population dose has been applied throughout CHI communities. The concept of population dose will continue to be refined, within CHI and beyond so that it becomes even more accessible to the many individuals, groups, and communities struggling with such dedication and intensity to improve the health outcomes of their families, neighbors, and colleagues. As this occurs, we welcome more communities joining the testing and refinement phase of this concept and how it can be measured.
Dose does matter. But what really matters is achieving the ideal of total health and wellbeing for as many of us as possible. Dose matters because it holds the promise of helping us get there faster and more efficiently and with more lasting results than would be the case if we did not have the insights dose can offer about what does and does not work.
Appendix A – CHI Logic Model
Each box and circle of this logic model has been explored by the ongoing CHI evaluation, but the main focus has been on the intermediate outcomes, specifically the health-promoting system changes that lead to increases in healthy eating (nutrition) and physical activity (shown in the red oval).
In addition, to better understand the context for interventions and the role of changes in community capacity, researchers have also focused on strategies that support capacity building at the community level.
Appendix B – Effect Size Groupings
(Based on strategy-level evaluation evidence and literature review. See Kaiser Permanente and Center for Community Health and Evaluation (September 2014), Measuring and increasing the “dose” of community health interventions. For more information, see http://share.kaiserpermanente.org/article/dose-creating-measuringimpact/)
For population dose purposes, “effect size” is synonymous with intervention strength. It refers to the degree to which people who are exposed to an intervention strategy change their behavior to make healthier choices as a result of that exposure. The “effect size” is the mathematical expression of the average percent change in behavior for each person exposed. For example, if participants in a produce cooking class ate four servings each day of fruits and vegetables before the class and five servings after taking the class, the effect size would be 25 percent (the one-serving difference between four servings before and five servings after the class, divided by the baseline of four servings before the class).
How do we figure out the strength or effect size? In general, these estimates come from either published research on a similar intervention and population or a specific evaluation of the intervention (as in the example above).
Table B-1 shows some effect size groupings based on strategy-level evaluations from CHI and literature reviews.
Appendix C – Strength Rating Guide
Appendix D – Strength Formulas by Strategy
Appendix E – Calculating Dose: A Physical Activity Strategy Example
A neighborhood park has added walking trails and a playground in an effort to increase physical activity levels for the neighborhood’s residents—those living within a quarter-mile radius of the park. The entire community has about 10,000 residents.
What Is the Strength Formula for Physical Activity Strategies?
The strength of a physical activity strategy reflects relative changes in the minutes of physical activity, either compared to baseline data from population surveys or the use of national averages for daily minutes of physical activity from the most recent National Health and Nutrition Examination Survey (NHANES) dataset. Estimates are adjusted for exposure—for example, the number of days children are in school or, in the example below, the number of days a park is usable because of local weather.
The formula would be:
Physical Activity Strategy Strength = (% impacted by the intervention) x (% increase in physical activity minutes) x (frequency of exposure)
What Do We Know (Or What Can We Learn)?
An evaluator observed that 10 people, on average, used the park per day at baseline, but after the improvements, 50 people used it every day. The park is located in a state where it can be used comfortably about nine months out of the year, weather permitting.
An additional intercept survey gleaned the information that users averaged about 30 minutes of physical activity per visit.
What Can We Safely Assume?
Estimates from NHANES and others suggest that the average minutes of physical activity per day for adults is 28 minutes per day, consistent with the intercept survey.
Using census data and mapping software, we are confident that about 500 people live within a quarter-mile of the park, and an intercept survey further confirms that nearly all users—90 percent—live this close.
Calculating Dose: A Fruit and Vegetable Strategy Example
A Community Supported Agriculture (CSA) program for low-income households provides free weekly deliveries of local produce to 100 families in a community of 10,000 for half the year—25 weeks. Each delivery includes about 10 pounds of produce.
What Is the Strength Formula for Fruit and Vegetable Strategies?
The formula reflects the relative change in servings of fruits and vegetables consumed, either compared to baseline data from a population survey, or the use of national averages for daily servings from NHANES datasets. Estimates are adjusted for exposure (e.g., the number of days students are in school and eating in a healthier cafeteria, the number of weeks during which fresh produce is delivered through a CSA program for low-income families).
The formula would be:
Fruit and Vegetable Strategy Strength = (% impacted by the intervention) x (% increase in servings of fruits and vegetables) x (frequency of exposure)
What Do We Know (Or What Can We Learn)?
A survey of the families indicates that users of this produce were eating about two cups of fruits and vegetables a day before the CSA program started.
About 10,000 people live in this community.
What Can We Safely Assume?
The average family size is 2.5 people. Each pound of produce is about 3 cups.
Calculating Dose: A Sugar-Sweetened Beverage Strategy Example
An elementary school whose enrollment is 800 students (out of a total district enrollment of 2,000) removes chocolate milk from the menu. Before the chocolate milk option was removed, about half the children purchased chocolate milk daily.
What Is the Strength Formula for Sugar-Sweetened Beverage Strategies?
The effect size of these strategies uses a formula that reflects the proportion of daily energy intake rather than the effect on calories per day (which varies significantly from one person to the next).
The formula would be:
Sugar-Sweetened Beverage Strategy Strength = (% impacted by the intervention) x (% energy intake affected) x (decrease in unhealthy) x (frequency)
What Do We Know (Or What Can We Learn)?
Sugar-sweetened beverages average 10 percent of total daily energy intake, with two beverages per day. Therefore, each sugar-sweetened beverage is equivalent to 5 percent of the total daily energy intake (per NHANES).
Half of the students purchased chocolate milk daily.
What Can We Safely Assume?
The removal is complete—no chocolate milk will be offered. Frequency of exposure needs to be adjusted for the number of days students are in school.
References
- Cheadle, A., S. Rauzon, R. Spring, P. M. Schwartz, S. Gee, E. Gonzalez, and D. Williamson. 2012. Kaiser Permanente’s Community Health Initiative in northern California: Evaluation findings and lessons learned. American Journal of Health Promotion 27(2):e59–e68. https://doi.org/10.4278/ajhp.111222-quan-462
- Cheadle, A., P. M. Schwartz, S. Rauzon, E. Bourcier, S. Senter, R. Spring, and W. L. Beery. 2013. Using the concept of “population dose” in planning and evaluating community-level obesity prevention initiatives. American Journal of Evaluation 34(1):71–84. https://doi.org/10.1177/1098214012458824
- Glasgow, R. E., L. M. Klesges, D. A. Dzewaltowski, P. A. Estabrooks, and T. M. Vogt. 2006. Evaluating the impact of health promotion programs: Using the RE-AIM framework to form summary measures for decision making involving complex issues. Health Education Research 21(5):688–694. https://doi.org/10.1093/her/cyl081
- Homer, J., B. Milstein, D. Labarthe, D. Orenstein, K. Wile, J. Trogdon, and P. Huang. 2010. Simulating and evaluating local interventions to improve cardiovascular health. Preventing Chronic Disease 7(1). Available at: https://pubmed.ncbi.nlm.nih.gov/20040233/ (accessed July 17, 2020).
