A Study Design to Explore
Service Coordination
in Home Visiting
A Study Design to Explore Service Coordination in Home Visiting
i
Table of Contents
Acknowledgments ................................................................................................................................................................................ 1
Introduction ............................................................................................................................................................................................ 2
Relevance of Topic ............................................................................................................................................................................... 4
Overview of Prior Work in this Topic Area ................................................................................................................................. 5
Research Questions ............................................................................................................................................................................. 7
Study A: Qualitative Examination .................................................................................................................................................. 8
Study B: Multilevel Modeling ........................................................................................................................................................ 18
Practical Considerations ................................................................................................................................................................. 32
Use of Findings ................................................................................................................................................................................... 34
Conclusions and Next Steps .......................................................................................................................................................... 35
Resources ............................................................................................................................................................................................. 37
References ........................................................................................................................................................................................... 38
A Study Design to Explore Service Coordination in Home Visiting
1
Acknowledgments
The authors would like to thank project consultants Deborah Daro, Jill Filene, Tamara Halle, Julia Heany,
and Jon Korfmacher for their review and contributions to this report. The authors would also like to thank
the following MIECHV awardees and other experts who helped conceptualize this study design: Gregory
Aarons, Clinton Boyd, Claire Canfield, Lucinda Caruso, Greer Cook, Evelyn Dryer, Anne Duggan, Terri
Enters, Felecia Fognani, Bryn Fortune, Angela Heisey, Carrie Higgins, Janet Horras, Gretchen Kirby,
Amanda Kocienski, Elli Komito, Lisa Martin, Steven Pascal, Sara Remington, Shannon Sainer, Sarah Schafer,
Christine Silva, Wendy Stafford, Kathleen Strader, Sophia Taula-Lieras, Angela Ward, and Nancy Whitesell.
The authors also thank the following project team members who contributed to this report: Kristina
Rosinsky, Mariel Sparr, Sarah Kelley, Beth Green, Chrishana M. Lloyd, and Lance Till.
A Study Design to Explore Service Coordination in Home Visiting
2
Introduction
Research suggests that high-quality implementation of
evidence-based home visiting programs increases the
odds of achieving the best outcomes for children and
families.
1
However, there is little consensus in the field
regarding the critical elements of home visiting program
implementation quality and how they may lead to
improved outcomes for families and children.
2
The
Measuring Implementation Quality in Maternal, Infant,
and Early Childhood Home Visiting (MIECHV)-Funded
Evidence-Based Home Visiting Programs project is a
collaboration between Child Trends and James Bell
Associates (“the research team”)—under the direction of
the Health Resources and Services Administration
(HRSA) and the Administration for Children and
Families (ACF)—that seeks to address these gaps.
For this project, the research team (the first-person
shorthand “we” will also refer to the research team)
conducted a literature review on what is known about
implementation quality.
a
The team also developed a
conceptual framework depicting the various factors that
are hypothesized to contribute to implementation
quality across levels of the home visiting system (e.g.,
family, home visitor, community context).
b
We engaged
MIECHV awardees and other home visiting experts
throughout this project to ensure our work is relevant
and applicable in the field. The final phase of this project
is the development of study design reports that outline potential research plans to address identified
awardee needs with respect to measuring program implementation quality. All of these study designs—
which represent a wide range of research questions, methods, and target audiences—are aimed at
deepening our understanding of the factors that may contribute to implementation quality in the home
visiting field.
a
Sparr, M., Goldberg, J., Thomson, A., Ryan, K., Kane, M., & Haas, M. (2021). Quality considerations across levels of the home visiting system: A
literature and measure review. Washington, DC: Health Resources and Services Administration, U.S. Department of Health and Human Services.
b
Crowne, S., Rosinsky, K., Goldberg, J., Sparr, M., Ulmen, K., and Huz, I. (2021). A conceptual framework for implementation quality in home
visiting. Washington, DC: Health Resources and Services Administration, U.S. Department of Health and Human Services.
The study design presented in this report examines one of the threads of implementation quality—
“coordination, collaboration, and relationship development”—featured in the conceptual framework our
team developed (see Figure 1). For this study, we focus on identifying which factors across the family, home
visitor, program, and community levels influence service coordination and predict successful service
connections for families.
Study Overview
Aim: To identify the supports and
activities that influence service
coordination in the home visiting
context and predict successful
connections to services for families
Design: Mixed methods descriptive
study
Data sources: Focus groups, surveys,
MIS data
Sample: Families, home visitors, and
program managers at LIAs within a
state
Technical skill level: Advanced
Estimated cost: Between $355,000 to
$676,000 for both studies
Estimated time needed: 2.5 years for
both studies
A Study Design to Explore Service Coordination in Home Visiting
3
Figure 1. Home Visiting Implementation Quality Conceptual Framework
Note: CQI stands for continuous quality improvement.
Source: Crowne, S., Rosinsky, K., Goldberg, J., Sparr, M., Ulmen, K., & Huz, I. (2021). A conceptual framework for
implementation quality in home visiting. Washington, DC: Health Resources and Services Administration, U.S. Department
of Health and Human Services.
The proposed research design includes two sequential components, Study A and Study B. Study A uses
qualitative methods to understand home visiting service coordination activities and anticipated outcomes
“on the ground,” from the perspective of families and home visitors. Study B tests the relationships of the
key constructs identified in Study A, using multilevel modeling to examine which aspects of service
coordination influence family connections to services (see the “key terms and definitions” box for key terms
and definitions used in this report). As will be explained in more detail below, the latter study component
builds on the first, and may not be appropriate or feasible for all awardees.
Key Terms and Definitions
Service coordination: This study uses the definition of service coordination posited by West et al. (2018):
“the deliberate organization of activities between two or more organizations to facilitate, in partnership
with the family, the delivery of the right services in the right setting at the right time.”
3
Service
coordination in home visiting entails much more than simply making a referral. It is complex and
multidimensional, with activities that support service coordination (see below) occurring at multiple levels of
the home visiting system, and likely influencing whether families are successfully connected to services.
A Study Design to Explore Service Coordination in Home Visiting
4
Supports and activities that influence service coordination (predictors of interest): These are the
elements, or factors, that may be necessary to support and implement service coordination for
families. For example, availability of community service providers to which families can be referred is
an element that supports coordination. Having a policy in place for when and how home visitors
should assess families for unmet needs could also be an element that supports coordination. In this
study, we will work with families and home visitors to operationalize the many types of activities that
support service coordination.
Family connection to service (outcome of interest): Family “connection” to services is also complex
and multidimensional, including, for instance, initial contact with providers, enrollment in services,
receipt of benefits, longer term engagement with services, etc. In this study, we will work with families
and home visitors to operationalize this outcome, co-creating an understanding of key aspects of
service connections that reflects their on-the-ground experiences. The study design in this report also
allows for the identification of family outcomes that may accrue from service coordination beyond the
benefits of coordination alone.
We begin by describing the topic’s relevance to the home visiting field, followed by a brief summary of prior
research in the area and research questions for both Study A and Study B. This is followed by study design
and methods for each study. We then explore practical considerations (such as required cost and expertise)
and explain how findings from the study may be used by home visiting programs to strengthen service
coordination efforts. We conclude with a summary of next steps for those interested in going beyond the
scope of the study presented here. This study design is one recommended approach to exploring the topic at
hand, and we encourage readers to consider ways to tailor the ideas presented to their own contexts,
interests, and needs.
This report is generally intended for MIECHV state awardee and tribal MIECHV grantee staff, researchers,
technical assistance providers, and model developers. This study design is presented based on the
assumption that a state awardee or tribal grantee would implement the study.
Relevance of Topic
Home visiting programs aim to support families across multiple domains, including positive parenting, family
health and well-being, social supports, child development, economic well-being, and educational
achievement. Depending on the home visiting model, programs either address these domains directly
through home visiting programming, or, when meeting certain needs is beyond the scope of the program, by
connecting families to resources and supports in their communities. Coordination with community service
providers is included in the MIECHV legislation as one of the program’s six benchmark domains, and
programs funded through MIECHV measure efforts to conduct screenings and referrals in the areas of
mental health, child development, tobacco, and substance use. However, most home visiting programs make
referrals to a broader array of service areas than those measured through MIECHV, including prenatal
health, housing supports, child care, and economic assistance.
4
Indeed, research has suggested that service
coordination is a key activity for many home visiting programs,
5
with home visitors often playing a role not
just in connecting participants to needed services, but also in helping them to navigate the services in which
they are already engaged.
6
Effectively linking families with needed services is a complex process that could be influenced by factors at
multiple levels of the home visiting system, such as: families’ interest in services, home visitor knowledge
about service availability and eligibility requirements, program supports and training for home visiting staff
around service coordination, availability and accessibility of services in the communities, collaborations
between home visiting programs and other community service providers, and state interagency
partnerships. While, as described below, there has been some work done in the home visiting field to
A Study Design to Explore Service Coordination in Home Visiting
5
describe what service coordination looks like in practice at various levels of the home visiting system, there
remain significant gaps in our understanding of this issue.
Overview of Prior Work in this Topic Area
In this section we briefly summarize studies that have examined service coordination in the field of home
visiting. The studies described below focus on the following: the extent to which community services are
perceived by home visiting staff as available and accessible to families, factors hypothesized to influence
service coordination, home visitor coordination practices, and family outcomes related to service
coordination.
Availability, accessibility, and effectiveness of community services
The Mother and Infant Home Visiting Program Evaluation (MIHOPE) assessed the availability, accessibility,
and effectiveness of nine community service types: prenatal care, family planning and reproductive health
care, substance use and mental health treatment, shelter for intimate partner violence, intimate partner
violence counseling/anger management, pediatric primary care, early intervention services, adult education
or employment services, and child care.
7
Of the 86 participating programs (representing 12 states), at least
80 percent of program managers reported that each of the nine services were available in their
communities, with most service types perceived as available by more than 90 percent of managers. There
was greater variability in program managers’ perceptions of whether these community services were
accessible (i.e., affordable, convenient, etc.) to families, with some service types (e.g., prenatal care, early
intervention services) perceived as accessible by around 70 percent of managers while others (e.g., child
care, intimate partner violence counseling/anger management, substance use and mental health treatment)
were perceived as accessible by only around half of the program managers. Similarly, while at least half of
program managers reported that providers were either “very” or “quite” effective for each of the nine
services, these ratings varied considerably depending on service type, with child care and intimate partner
violence services again rated by fewer managers as effective than other services.
MIHOPE also examined home visiting programs’ use of memoranda of understanding (MOU) to formalize
service referrals, having a point-of-contact with the service provider, and having a rating identifying
coordination with the service provider as “good” or “excellent.”
8
Researchers found that fewer than half of
program managers reported having an MOU in place with a service provider for any of the nine services,
approximately one- to two-thirds of program managers reported having a designated point of contact with
at least one service provider, and more than half reported having a “good” or “excellent” coordination rating
with at least one service provider.
Factors that influence service coordination
The complex and multi-faceted nature of service coordination is reflected in a framework developed by
West et al. (2018) with input from home visiting experts.
9
The measurement framework for service
coordination includes multilevel inputs, outputs, and outcomes related to home visiting service
coordination. Inputs are organized into four categories: (1) national, state, and local context (e.g., leadership
and infrastructure), (2) families (e.g., service needs and preferences), (3) organizational context (e.g.,
relationships with other organizations), and (4) implementation system (e.g., training and supervision).
Outputs, or activities, comprise indicators within seven subdomains:
1. Establish roles across organizations
2. Assess family strengths and needs
3. Create a goal plan
A Study Design to Explore Service Coordination in Home Visiting
6
4. Facilitate referrals and linkages
5. Monitor, follow-up, and respond to change
6. Support self-management of goals
7. Align services with population needs and community resources
And finally, the framework includes short-term program outcomes (e.g., increased family engagement in
home visiting, increased referrals), short-term systems outcomes (e.g., improved relationships between
home visiting and other community providers, decreased unmet needs), and long-term outcomes (e.g.,
improved life course trajectories).
The authors of this framework have used it to inform a series of studies examining service coordination at
home visiting sites across the country. In their study of state-level perceptions of service coordination,
West, et al. (2020) found that states and territories varied widely in the extent to which service coordination
supports were in place at the state levels, with MIECHV leadership emerging as a relative area of strength,
while data systems and finance were rated as less robust by most respondents.
10
In this study, reported
barriers to coordination included a lack of a shared data system, insufficient funding, and the perception
that coordination was not a high priority for others. In a study of service coordination at the local
implementing agency (LIA) level, West et al. (2021) surveyed home visitors about the strength of local sites’
implementation systems (e.g., supervision and training) and coordination activities (e.g., screening and warm
handoffs for referrals) with a focus on three service types: mental health, intimate partner violence, and
substance use.
11
Results from this study suggest that sites felt they were better equipped to support families
with needs related to mental health and intimate partner violence than substance use. Implementation
system scores were also higher for screening and referrals than subsequent linkages and follow-ups, with
scores lowest for offering a warm handoff. The barriers to successful coordination that were most
consistently reported at the site level were limited availability of and access to services.
Examining home visitor service coordination practices through
management information systems
Goldberg et al. (2018) conducted a study using data from a statewide management information system
(MIS) to document the range of home visitors’ service coordination practices with a sample of families over a
four-year period.
12
The authors coded and analyzed 11,096 home visiting records, focusing on identifying
referrals, connections, disconnections (i.e., discontinued or interrupted services), and home visitor
coordination activities across multiple service areas (e.g., housing, mental health, child care, substance use
treatment). They used a multilevel scheme capturing all stages of home visitors’ facilitation of participants’
linkages to community services, including pre-referral activities (e.g., suggesting a service), referrals (i.e., the
initial action taken to link a participant to a service), referral follow-up activities (e.g., assistance completing
applications), service connection, service disconnection, post-connection activities (e.g., satisfaction check-
ins), and post-disconnection activities (e.g., attempts to re-engage). The authors identified a hierarchy of the
intensity of time and effort required from home visitors in providing each type of support, including low-
level support (check-ins), moderate support (encouragement/suggestions/advice; emotional
support/cheerleading; information provision), and advanced support (instrumental support [e.g., helping
participants complete applications]; interagency case review). Results from this study suggest that home
visitors engage in multiple activities related to service coordination beyond the initial referral, from lower-
level supports such as checking in and reminding families to follow-up with a provider to intensive supports
such as completing applications, accompanying families to appointments, and engaging in interagency case
reviews. The authors found that, beyond providing contact information, helping participants make calls, and
following up on referrals, home visitors also helped to mitigate barriers associated with transportation,
logistical conflicts, or with emotional or psychological challenges related to service engagement (e.g.,
disrespectful provider).
A Study Design to Explore Service Coordination in Home Visiting
7
Family connections to services
While there have been some recent studies examining the ways in which service coordination is
implemented in home visiting, research examining service coordination outcomes remains relatively sparse.
Studies that looked at families’ service outcomes largely focused on whether an initial connection to a
service occurred following a referral. Taken together, these findings suggest that evidence-based home
visiting with an explicit focus on service coordination as part of their programming (e.g., Child First, Family
Connects, and SafeCare Augmented) are successful in facilitating initial connections to community
services.
13, 14,15
Findings from studies of evidence-based home visiting without this programmatic focus are
less consistent; Oregon Healthy Families America, for example, successfully linked families to services, while
Hawaii Healthy Families America was found to have no effect on service linkages.
16,
17
It should be noted,
however, that none of these studies examined whether the participants’ connections to community services
were direct results of home visiting referrals; rather, they looked at the program group’s general use of
services in comparison with those of the control group. One study that did examine the pathways that lead
from referral to connection found that only 21 percent of referrals resulted in a connection to services
(defined as initial contact with, enrollment in, and/or receipt of service/benefit) with connections in some
service areas (e.g., housing) requiring intensive service coordination efforts on the part of home visitors.
18
Research Questions
The two-part mixed methods study proposed in this report is guided by the following research questions,
organized by study component:
Study A. Qualitative Study
A.1. What supports and activities, at the community, LIA/program, home visitor, and family levels of
the home visiting system, do home visitors and families believe contribute to effective service
coordination?
A.2. How do home visitors and families define success related to home visiting service coordination?
What do they see as the key outcome(s) of these supports and activities?
Study B. Quantitative Study
B.1. Which service coordination supports and activities, at the community,
c
LIA/program, home visitor,
and family levels of the home visiting system are most predictive of family connection to services?
c
Note that community and program variables will be analyzed at the same level. There is typically one home visiting program per
community; thus, it is not possible to separate communities and programs into distinct levels.
B.2. Do the service coordination supports and activities identified as predictors in question B.1. still
drive family connection to services when controlling for community, LIA/program, home visitor, and
family characteristics?
B.3. To what extent are associations between service coordination supports and activities and families’
connection to services moderated by community, LIA/program, home visitor, and family
characteristics?
Both study components are aimed at generating a more comprehensive understanding of key predictors of
service coordination and the family-level outcomes that result from these service coordination predictors.
The goal of Study A is to work with home visitors and families to (1) learn how relevant the key constructs
A Study Design to Explore Service Coordination in Home Visiting
8
identified in the literature are to their lived experiences with service coordination; (2) further define and
operationalize these key constructs; and (3) uncover other supports, activities, and outcomes that may be
more salient and important to study participants than those established in the literature. The goal of Study B
is to use some of the predictors and outcomes that are operationalized and honed in Study A in a multilevel
analysis of family service coordination outcomes (see Figure 2). Note that the purpose of this report is to
describe Study A and Study B. While we touch upon the steps required to move from A to B (the green
arrow in Figure 2) in the methods section, a full description of those activities is beyond the scope of this
project.
Figure 2. Coordination Study Design Components
Informed by work from Duggan et al. (2018), West et al. (2018, 2021), and Goldberg et al. (2018),
19,20,21,22
we view the supports and activities that influence service coordination (i.e., hypothesized predictors) as
multidimensional, comprising four categories of indicators:
1. Community service indicators (e.g., availability and accessibility of service providers),
2. Implementation system indicators (e.g., MOUs, co-location, policies, and other interagency
agreements),
3. Collaboration indicators (e.g., relationship-building with providers, points of contact), and
4. Coordination activity indicators (e.g., referrals, follow-ups, and other home visitor behaviors in support
of coordination).
Both the qualitative and quantitative components of this design are organized around these four categories,
as described in the sections that follow. We now turn to the study designs, beginning with Study A.
Study A: Qualitative Examination
Research design
For Study A, we are proposing a qualitative design comprising focus groups with home visiting staff and
families to generate a comprehensive understanding of what service coordination looks like in practice.
Previous research on service coordination in home visiting has largely relied on key constructs and
indicators that were generated a priori by researchers and other national experts from the field; to date,
there have been no studies examining how service coordination in home visiting is seen and understood by
A Study Design to Explore Service Coordination in Home Visiting
9
those who are closest to it. The qualitative design proposed here takes into account the full complexity of
the subject, asking home visitors and families who are participating in home visiting about their community
systems of care, the ways in which families access services, and the role home visiting programs play in
helping families navigate these systems. It also asks home visitors and families to define what they see as
successful outcomes from home visiting service coordination activities, and to hypothesize what promotes
these outcomes.
This qualitative approach will allow researchers to operationalize key constructs for use in future research,
including but not limited to the study design proposed in Study B. Working with families and home visitors
to develop and refine these key constructs will be an important foundation for future studies of service
coordination in home visiting settings. Awardees hoping to use research findings to improve service
coordination in their state, tribe, or territory will benefit from using indicators that already have been
identified as salient to home visitors and families within those specific local contexts.
Samples
For this study, we recommend a combination of purposive and convenience sampling (see the “sampling
approaches” box for definitions) of home visitors and families
d
who are participating in home visiting. The
sample for this study, which is designed to be implemented by MIECHV awardees, should be drawn from
five or six program sites. This number could be more or less, depending on the size and diversity of the state,
tribe, or territory. Ideally, the program sites should be purposively selected by researchers in collaboration
with MIECHV administrative staff and should be representative of the demographics (i.e., racial, ethnic,
socio-economic), geography (i.e., urban, suburban, rural, frontier), and community service environments (i.e.,
resource rich, resource poor) that characterize the home visiting system in which the research is being
conducted.
d
By “families” we mean individual caregivers who are participating in home visiting. We use the terms “families” and “caregivers”
interchangeably in this section.
We recommend two focus groups at each program site: one with home visitors and one with caregivers,
each with six to eight participants,
23
for a total of 60 to 96 participants across all program sites. Given the
complexity of the subject matter, we are suggesting focus groups be kept small. However, this is a suggested
sample size only: awardees could feasibly narrow or broaden the scale depending on interest and resources
without jeopardizing the robustness of the design.
Focus group participants will likely be identified through convenience sampling. To recruit home visitors, we
recommend inviting all home visitors to participate in the focus groups, with the expectation that not all will
accept and/or the timing will not work for everyone. If more than eight home visitors express interest and
availability, we recommend holding two smaller focus groups at that site, since including more than eight
people may make the conversations less nimble and effective. To recruit caregivers, a good strategy is to ask
program supervisors and home visitors to recommend families they think would be likely to participate, and
either have the researchers reach out directly, or ask home visiting staff to contact them on the researchers’
behalf. Again, if more than eight individuals from one program site express interest, it is best to hold more
than one focus group.
Just as we recommend deliberately choosing program sites that represent the overall demographics of the
home visiting system, it is similarly critical to consider linguistic and cultural representation when recruiting
caregivers. For instance, if the majority of families in a program site are Spanish-speaking, the caregiver
focus group should be conducted in Spanish with majority Spanish-speaking caregivers, if at all possible.
A Study Design to Explore Service Coordination in Home Visiting
10
Research teams should include members that reflect the linguistic and cultural diversity of the program
sites participating in the study. Ideally, research teams should also include members that have some
familiarity with the community service environments in which the programs sit. Knowing who the key
players are in a community will make both facilitation and note-taking easier for the research team.
Sampling Approaches
A purposive sampling approach identifies individuals with particular characteristics that are most
relevant to the questions at hand. In this case, you may want to purposively sample from program sites
in areas with limited community resources, and from resource-rich environments.
A convenience sampling approach identifies individuals who are most accessible to you and willing to
participate in an interview. For instance, if you are hoping to speak with family members, you may ask
home visiting programs to distribute an email asking for those who are interested in participating in the
study.
A random sampling approach identifies the universe of individuals you may want to speak to and
randomly selects the individuals to participate. For example, you may have a list of all home visitors and
randomly select five home visitors to invite to an interview.
Data sources
Data sources for Study A include 1) focus groups with home visitors and 2) focus groups with caregivers.
Topic guides for these data collection activities use the predictor categories previously described
(community service indicators, implementation system indicators, collaboration indicators, and
coordination activity indicators) as a structure (see Table 1 for key constructs and example questions).
Home visitor focus groups
Focus groups with home visitors should be no longer than 90 minutes and should include one facilitator and
one notetaker.
Focus groups with home visitors will cover the following topics:
• Community service landscape for families with young children, including the availability and
accessibility of community services; how well service providers in that community coordinate with each
other to support families with young children; and the extent to which the home visiting programs share
service coordination infrastructures with other community providers, such as shared data systems and
funding.
• Program implementation system, including the policies, procedures, and expectations in place to
support service coordination; the types of service coordination-focused training and supervision
available to home visitors, and how well-equipped home visitors feel to do this work; how the program
keeps track of service coordination activities and whether those systems are shared with other
community providers; and the extent to which programs financially invest in service coordination
activities.
• Collaboration activities, including structural factors (e.g., the presence of formal agreements and
designated points of contact with other community service providers), participation factors (e.g. does
the program participate in local coalitions, community events, advisory boards?), relational elements of
coordination (e.g., the more informal partnerships programs have with other community service
A Study Design to Explore Service Coordination in Home Visiting
11
providers and the extent to which other providers understand the mission and services of the home
visiting program), and the extent to which home visitors believe these supports influence effective
service coordination.
• Coordination activities, including how they typically assess families for their service needs; what types
of activities they engage in when making referrals; how they follow up on referrals; how they support
families’ involvement in other services and programs; and, more generally, what role they see
themselves playing in helping families navigate the local systems of care, and how they believe that role
could be strengthened.
• Family outcomes, including questions about how they define success related to their service
coordination, how they think families benefit from these activities, what supports and activities home
visitors see as leading to effective service coordination, and what could be done to improve the system
of care to better meet families’ needs and preferences.
Some home visitors may feel inhibited or cautious about sharing opinions about other service providers
during the group. It is important for the facilitator to remind participants at the beginning of the focus group
about the importance of ensuring that everything said during the discussion remains strictly confidential. In
addition, the facilitator should let participants know that they do not have to talk about specific providers,
families, or programs; rather, they should feel free to speak in general terms or use pseudonyms if they feel
more comfortable.
Family focus groups
Family focus groups are oriented around an activity developed by Goldberg and Litovich titled Family
Network Mapping, and typically last between 90 to 120 minutes.
24
Caregivers are guided through a process
using paper, post-it notes, and stickers (see the “Family Network Mapping” box for an activity description),
in which they document the services they receive (including both formal and informal supports families
receive from MIECHV-funded programs), their experiences with these services, gaps in coordination
between services, and the role home visiting has played in these personal systems of care. The family focus
groups will specifically probe on the types of coordination activities (e.g., referrals, follow-ups) their home
visitors have engaged in with them.
During the activity, participants will be engaged in a discussion about what their personal service networks
look like and the role that their home visitor has played in helping them navigate these service networks.
They also will be asked about what they want from home visiting in the way of service coordination,
including what types of assistance they want, and how their preferences differ depending on the type of
service. Finally, families will be engaged in a discussion about what they see as ideal outcomes from home
visiting service coordination activities, how they define their own success in terms of service connections,
and what other outcomes they think result (or could result) from home visitors’ service coordination efforts.
As is the case with the home visitor focus groups, the facilitator should remind participants of the
importance of respecting each other’s privacy and confidentiality. Participants should also be told that if
they are uncomfortable specifying the name or service type of any particular program on their map, they
should feel free to either skip that service/program or give it an alternative label. In other words, they can
provide as much or as little detail as they like on their Family Network Maps. The facilitator should
emphasize that the goal of the evaluation is not to learn about every single program families are involved
with, but rather to learn about how families access services more generally, and the role home visiting can
play in advancing that access. In our experience, these groups are best held in person; doing the project
together in the same room helps the group cohere, and helps the conversation feel less formal and flow
better. If groups need to be conducted virtually, there are two options: 1) supply participants with the
materials they will need (e.g., poster board, stickers) ahead of the group, or 2) use Mural, JamBoard, or some
other type of digital whiteboard application.
A Study Design to Explore Service Coordination in Home Visiting
12
Family Network Mapping
The activity requires flip chart paper, post-its, and stickers, and comprises the following steps for
participants to follow:
• As a group, brainstorm services/resources in the community that participants might use or access
(e.g., food stamps, education support, child care, job training, mental health care), and record them
on a flip chart in the front of the room.
• Each participant writes every service they are involved with on post-it notes, which they then place
on their own individual poster papers. [Remind participants to reference the list the group brainstormed
together to jog their memories about the services they are involved with].
• Participants use color-coded stickers to indicate their overall feelings about each service—green for
mostly positive, red for mostly negative, and yellow for neutral. Encourage participants to “go with
their gut” for this part, prompting them to think about things like how much of a hassle it was to
access the services, what kind of interactions they have had with the caseworker, whether they get
timely call-backs, etc. They are told that they can put more than one sticker on a service to indicate
strong feelings, and also can put more than one color on a service to indicate mixed feelings. [During
this activity, engage participants in a discussion about the key barriers to accessing certain services, what
made it hard to access services, what types of things made it easier, and what kind of supports they wish
they had].
• Participants add a purple sticker to every service that their home visitor has helped them with in any
way (e.g., referring them, helping them access the service, helping them troubleshoot problems, etc.).
[During this activity, engage participants in a discussion about the role home visitors have played in their
service coordination, including what activities helped the most, and what participants see as the key
outcomes of home visitor activities. Ask about their connections to services, while also probing for other
outcomes like learning to advocate for themselves, being better able to support their peers, and being better
able to navigate the system themselves, etc.].
• Participants draw lines between service providers that, as far as they know, communicate with each
other about their case.
• Participants draw a pie chart to indicate how much time they spend talking about their service
needs with their home visitor and how much time they spend talking about parenting and child
development.
A Study Design to Explore Service Coordination in Home Visiting
13
Table 1. Indicator Category, Key Constructs, and Example Questions Mapped onto Data Source
Indicator
Category
Example Questions*
Home
Visitor
Focus
Groups
Family Focus
Groups (with
Family
Network
Mapping)
Community
Service
Indicators
Do you feel like your community has sufficient services and resources to meet
families’ needs? Do you know where to go for [service]? What services are you/your
families currently involved with? What types of services does your community need
more of?
x
x
When families are referred to [service], what do you think are their chances of
getting served? What types of barriers [probe for logistical (e.g., transportation,
eligibility, waitlists), structural (e.g., systemic racism), and psychological (e.g.,
disrespectful providers, non-trauma-informed)] get in the way? In general, how well
do you feel like this community supports families with young children?
x
x
What coalitions or partnerships focused on supporting families exist in this
community? How well do local programs work together to support families? Are
services streamlined or duplicative [probe for eligibility requirements, same
screens, etc.]? What would make coordination among providers work better?
x
Do you attend trainings with staff from other programs/sectors? What other kinds
of overlaps exist among service providers in your community [e.g., probe for shared
competencies, data systems, funding]?
x
Implementation
System
Indicators
What policies/expectations for service coordination are in place at your program?
What types of service coordination activities does the program expect of home
visitors [probe for expectations around screenings and assessments, referral-
making, and follow-up activities]? Does your program have dedicated staff for
service coordination? If not, would that helpful?
x
Is there training on service coordination provided by your program? Is service
coordination discussed during supervision? Is there adequate time in home visitors’
schedules for service coordination activities? Do you feel like you/home visitors are
well-equipped to do the work of service coordination? What supports are needed?
x
How does your program keep track of referrals and follow-up activities? Does your
program have any shared tracking systems with other community service
providers? Are/would shared tracking systems be helpful in supporting families’
connections to services and making your work easier?
x
A Study Design to Explore Service Coordination in Home Visiting
14
Indicator
Category
Example Questions*
Home
Visitor
Focus
Groups
Family Focus
Groups (with
Family
Network
Mapping)
What kinds of investments does your program make in service coordination [e.g.,
probe for dedicated funding for coordination initiatives, incentives for staff to sit on
advisory boards]? Is this sufficient? What more is needed?
x
Collaboration
Indicators
Is your program physically located with any other agencies or programs? Does your
program have any memoranda of understanding (MOUs) or other types of formal
agreements with other community programs? Do you have designated points of
contact at [service]? Does your program share an intake system with other
programs? How do the above factors influence service coordination at your
program? Are there other systems you think would strengthen collaboration and
make your work easier?
x
Do staff from your program sit on advisory boards or working groups for other
programs? What about staff from other programs on your advisory boards? Are
staff from your program members of any community coalitions or initiatives? What
about community events, etc.? How do you think these activities influence service
coordination at your program?
x
How would you characterize your relationship with other service providers in the
community [probe for specific services]? Do other providers understand home
visiting and its role in the local system of care? Is home visiting respected? Do you
have informal relationships/experiences with other community service providers
that affect your ability to connect families to services?
x
Coordination
Activity
Indicators
Thinking about an average visit, about how much time is spent in talking about other
service needs and how much time is spent talking about parenting and child
development? [Ask the same question about workload for home visitors only].
x
x
How are families assessed for service needs [probe for screenings, intakes, follow-
ups, etc.]?
x
x
How do home visitors typically make referrals when families need to be connected
with a service [probe for providing information, helping families make the first call,
calling for the families, filling out applications with families]? How does this differ by
service area? Which activities do home visitors and families feel are most effective?
x
x
How do home visitors typically follow-up with families once the referral has been
made [probe for checking in with families, reminding families to call back, calling the
providers themselves, going with families to programs]? Which activities do home
visitors and families feel are most effective?
x
x
A Study Design to Explore Service Coordination in Home Visiting
15
Indicator
Category
Example Questions*
Home
Visitor
Focus
Groups
Family Focus
Groups (with
Family
Network
Mapping)
Thinking about the services families are already connected to, what role, if any, do
home visitors play in helping families maintain those service connections? What
activities do families prefer and how do those preferences vary based on service
area?
x
x
Family Service
Outcomes
What constitutes a successful outcome from service coordination activities? How
do these outcomes vary depending on the type of service home visitors are focused
on [probe for differences between benefits and one-shot services (e.g., enrollment in
WIC, receipt of cash assistance or child care subsidies) and longer-term services
(e.g., early intervention, therapy)]? Are there outcomes other than service
connections that are salient to families [probe for things like family ability to self-
advocate, persistence, knowledge about community services]?
x
x
*The questions listed here are illustrative examples only: the focus group topic guides should be tailored to the community contexts in which the programs sit. Additionally, these questions
are phrased more generically than they would be in practice in order to be applicable across the different data sources listed. For instance, “Do you feel like your community has sufficient
services and resources to meet families’ needs?” would be phrased differently when asked of families (e.g., “Does this community have the services you need?”) and home visitors (e.g., “Do
you feel like this community has enough services and resources to meet the needs of the families you work with?”)
A Study Design to Explore Service Coordination in Home Visiting
16
Analysis plan
We recommend having two study team members in each interview and focus group—one who facilitates the
conversation and one who is responsible for taking notes. We suggest recording and transcribing the
sessions, if possible. Having a recording to refer to when summarizing notes from each session can be very
helpful. Recordings also are helpful (although are not strictly necessary) when using software such as
Dedoose or NVivo for coding and analysis. If recording is not possible for some reason (e.g., participants do
not consent to be recorded, or recording technology is not available), detailed notes that capture all points
made during the conversation may be sufficient, assuming verbatim quotes are not needed.
To analyze focus group data, we recommend
a combination of deductive and inductive
coding approaches (see the “coding
approaches” box for definitions), as explained
in the steps below. We propose this approach
because the goal of Study A is both to
validate and operationalize the key
constructs that already have been put forth
in the literature, but also to better
understand what matters to people closest to
the work, which may be different from, or
above and beyond, the constructs already
identified in the field.
Coding Approaches
Deductive coding is top-down approach where you
start by developing an initial set of codes based on
an existing research framework or theory.
Inductive coding uses a ground-up approach where,
rather than beginning with pre-conceived ideas
about how to organize the data, the codes emerge
from the raw data itself.
1. Generate and assign initial codes based on the indicator categories and key
constructs guiding the research
These broad starting codes should be based on the categories and key constructs outlined in Table 1. For
example, if a focus group participant comments that there are not enough mental health providers in the
community, that piece of text would be coded deductively as “1. Community service indicators” and “2.
Availability of community service providers.”
2. Generate and assign codes within each key construct
Once the text has been divided into these broader codes, more specific codes within each key construct area
should be assigned. Using the example from above, the text would be given a third code such as “insufficient
mental health.” This step could also be done in Step 1 as part of the higher-level coding. We recommend a
combination of deductive and inductive coding for this step, as follows:
Deductive Coding
To generate codes for deductive coding, we recommend first drawing from the research informing this
project. For community service indicators, implementation system indicators, and collaboration indicators,
researchers could refer to the Home Visiting Service Coordination within the Early Childhood Systems
survey
e
developed by West et al. for use in their study of site-level home visiting service coordination
(2021).
25
See below examples:
e
While this survey has not yet been made publicly available; we recommend contacting lead author Allison West at Johns Hopkins
Bloomberg School of Public Health for inquiries related to its use.
• When focus group participants describe barriers to accessing services (within the key construct,
“accessibility to community services”), initial coding of these text blocks could be based on the response
A Study Design to Explore Service Coordination in Home Visiting
17
items for the survey question “What barriers make it hard for families to access [resources…],” which
are as follows: No slots available/wait list, Cost of service is too high, Location or transportation,
Services only offered during work hours, Lack of child care, Services not available in families’ primary
language, and Families don’t meet eligibility requirements.
• For coordination activity indicators, researchers could refer to the coding scheme developed by
Goldberg et al. (2018) to describe home visitor practices related to service coordination. For example,
when focus group participants talk about the activities their home visitor engaged in to help them
secure housing, activities could be coded as Encouragement, suggestions, and advice; Emotional support/
cheerleading; Information provision; Instrumental support; or Interagency case review.
26
Inductive Coding
While the focus group topic guides are structured according to the aforementioned theoretical frameworks,
it is also likely that new and different themes—specific to the programs’ particular contexts and practices—
will emerge from the data. Therefore, we recommend the use of thematic coding of text to supplement the a
priori codes based on the research frameworks. For instance, returning to the first example above, home
visitors may identify service barriers not accounted for, and considered more salient than those in the West
et al. (2018) framework,
27
such as structural racism and community violence. And families may observe that
one of the most important outcomes of service coordination for them is increased confidence in seeking out
services, rather than being connected to the service itself. Finally, though not specifically accounted for in
the framework, any family, home visitor, program, and community characteristics that emerge as
particularly salient during the focus groups should be included in the coding scheme. This inductive coding
process is a critical complement to the deductive process outlined above.
3. Summarize themes within and across focus groups
The final step is to summarize the themes that emerged within and across focus groups. Findings from this
research could be presented descriptively to summarize how home visiting systems support service
coordination, including key supports and activities, and expected outcomes of service coordination efforts.
To use findings from Study A to inform variables and outcomes to include in the multilevel modeling
proposed in Study B, the final codes should also be organized according to the categories presented in Table
2, below. (Note that the indicator categories in Tables 1 and 2 are the same). Specifically, codes should be
considered for their relevance to:
• Family service connection outcomes
• Community service indicators
• Implementation system indicators
• Collaboration indicators
• Coordination activity indicators
• LIA/program characteristics
• Home visitor characteristics
• Family characteristics
Use of findings from Study A
As described previously, some awardees may choose to only complete Study A. Completing Study A will
provide information about how service coordination in home visiting is seen and understood by home
A Study Design to Explore Service Coordination in Home Visiting
18
visitors and families in their states, tribes, or territories. Results from the focus groups can be shared with
program staff and administrators to highlight where there is alignment between home visitors and families
in how they view service coordination activities and outcomes and where there are discrepancies. They also
may help home visiting administrators identify where there are barriers, challenges, and successes in service
coordination; this information may then be used to improve service delivery, training, and technical
assistance focused on service coordination. In addition, this information could be used to build and
strengthen relationships with other community service providers. For example, as part of examining
collaboration indicators, home visitors may identify specific strategies they feel would improve service
coordination with partnering agencies. See more discussion of ways to use Study A results in the general
Use of Findings section at the end of the report.
Study B: Multilevel Modeling
Research design
We propose a quantitative study design using multilevel modeling (MLM) of survey and program data from
multiple programs in a single state, tribe, or territory.
f
MLM is a good approach to use when looking at
outcomes using a set of predictor variables clustered into nested levels. In MLM, the lowest level represents
the most detailed unit of analysis and has the highest number of data points. In this case, we are interested
in predicting the family service outcome using a three-level model, in which families (Level 1) are clustered
within home visitors (Level 2), and home visitors (Level 2) are clustered within programs/communities (Level
3).
f
Design adapted from Fauth, R. C., Latimore, A., Kelley, S., Goldberg, J., & Supplee, L. (2020). MIECHV program: developing strategies to
facilitate Cross-Model Collaboration and Data Sharing (MODS): Research designs report. Rockville, MD: Health Resources & Services
Administration, U.S. Department of Health and Human Services.
MLM assesses which factors account for the most variability in the outcome of interest—in this case family
connection to service—and at which level (i.e., family, home visitor, or program/community). It also allows
for exploration of whether the associations between the supports and activities that influence service
coordination (predictor) and service connection (outcome) are the same for everyone, or whether they vary
according to families’ home visitors or by home visitors’ programs.
The indicator categories and key constructs we propose for use in this model mirror those outlined in Study
A; community service indicators, implementation system indicators, collaboration indicators, and
coordination activities. Also included in this model are characteristics of community/programs, home
visitors, and families. Finally, the outcome variable of interest is family connection to a community service
g
(see Figure 3 for the nested model).
g
Note that while we use “family connection to a community service” as the primary outcome throughout this section, this variable can
be substituted with constructs that emerged during Study A as meaningful for families and home visitors.
A Study Design to Explore Service Coordination in Home Visiting
19
Figure 3. Study B. Multilevel Model of Service Coordination
Note: The primary outcome of interest, shown here as “Family connection to a community service,” can be substituted with
outcomes that emerge during Study A as meaningful for families and home visitors.
There are many advantages associated with multilevel modeling. First, as alluded to above, it helps address
the “problem” of having nested data, or data points that are not independent. Separating the levels is
important because there are factors that affect family connection to services that can be attributed to the
program families attend or which home visitor they have. Further, it is likely that families with the same
home visitor attending the same program in the same community share some experiences that influence
service connections, making them more similar to one another than to families with different home visitors
working in different programs in different communities. MLM allows for data to be examined at one level
while accounting for the variance at other levels. This quantitative approach will also allow researchers to
test and isolate key aspects of coordination for targeted examination in future research. For example,
researchers may want to examine how the relationships found in this study vary according to service area
(e.g., mental health, child care).
There are several considerations awardees should weigh before implementing this study design, as detailed
below.
Considerations for using this approach
MLM is a complex design and may require samples, data sources, and analytic expertise that some states,
tribes, and territories may not have:
• Data availability. As described in more detail below, the data source we advise for the referral activities
and family service outcomes is a web-based management information system (MIS). This is because
while the community service indicators, implementation system indicators, and collaboration indicators
can be assessed through one-time surveys of home visiting providers, the activity indicators and family
service outcomes need to be assessed over time as part of home visitors’ daily practice. The least
burdensome and most accurate way for this data collection to occur is through an existing MIS. We do
know from previous research that there is tremendous variability in how home visiting programs collect
and track data on screenings, referrals, participants’ service connections, and community service
A Study Design to Explore Service Coordination in Home Visiting
20
enrollments, with some programs using paper records rather than web-based systems.
28
Study B may
not be appropriate for home visiting programs that do not have a robust MIS already in place.
• Sample size. Since the absolute minimum number of program sites required for this study is 10 (and
ideally should be closer to 20), this design is more appropriate for larger state home visiting systems
(more on sample size in the next section).
• Level of difficulty. Conducting MLM requires considerable computational skill and analytic expertise.
Researchers should consider this skillset when putting together their teams.
On the other hand, if it is not possible to implement the design as written here, for the above or other
reasons, this study can be adapted to be more descriptive and exploratory. For instance, if programs do not
have a robust MIS, family surveys could be used to measure coordination activities and outcomes. Also,
awardees with fewer than 10 program sites could use the methods proposed here to conduct a descriptive
quantitative study of service coordination, without the final step of multilevel analysis.
Sample
Having a sufficient sample is key for this design. In addition to data from the programs’ MIS, data will be
collected via surveys of home visiting supervisors and home visitors. Participant samples should be drawn
from 10 to 15
h
program sites, as follows:
h
While an ideal sample size for program site is closer to 20, we recognize that many MIECHV awardees will not be able to meet this
criterion and we are therefore recommending a more feasible number of sites.
• 1 program manager per program site, for a total of 10-15 program managers
• 5-15 home visitors per program site, for a total of 50-225 home visitors
• 5-15 families per home visitor, for a total of 250-3,375 families
Because these sample sizes are so large, we recommend census sampling for the family, program manager,
and home visiting staff surveys (e.g., they should be distributed to all home visiting staff in a program). The
remaining data come from MIS, which already collects data for all families enrolled in programs.
Conducting power analysis
A power analysis can indicate the sample size needed to detect a hypothesized relationship. We have
deliberately designed this study to be open-ended, meaning that the variables used at each level of the
model will depend on the specific set of hypotheses each state decides to pursue. Because we do not have a
precise analytic model, we are unable to conduct a power analysis for this report.
29,
30
States will need the
following to estimate the required sample:
• A precise model specification (i.e., exactly which variables will be included in the multilevel model)
• A specific hypothesis (i.e., which model parameter is the focal point of the analysis)
• An estimate of the mean and standard deviation (or the proportion for binary variables) for outcome
and predictor variables in the model
• An estimate of the expected relationship size between the predictor and outcome variables
i
• An estimate of the intraclass correlations (i.e., what proportion of the outcome variation is at the two
higher levels of the model)
i
This is assuming we seek to calculate sample size. One could also do the power analysis with a known sample size to calculate the
minimum detectable effect size.
A Study Design to Explore Service Coordination in Home Visiting
21
Once the precise model design has been specified, a power analysis should be used to select an appropriate
sample size. Power analysis for three-level multilevel models is more complex than it is for single level
designs because it relies on simulations. We recommend the use of the MLPowerSim software, which can be
used to generate R (or MLWiN) code to run such simulations and derive estimated power.
31
The associated
guide
32
has detailed instructions on how to input model parameters into the program to produce the desired
power estimates. We suggest, as is standard in the field, aiming for at least 80 percent power to detect the
target effect.
33
Data sources
Data sources for this study will include a (1) program’s MIS, (2) LIA/program manager survey, (3) home
visitor survey, and 4) family survey. Each is described in more detail below. See Table 2 for indicator types
and key constructs mapped onto MLM levels, data sources, and research questions. In this section, we
suggest key constructs and example indicators. These should be seen as examples only; the final set of
indicators chosen for this study will depend on which aspects of coordination and/or which service areas
awardees are interested in, as well as findings from Study A. For example, implementers of the study may
want to focus on coordination for only one service area, such as mental health. In this example, the
community service indicators may be limited to behavioral health providers; implementation system
indicators may be limited to constructs such as the presence of a clinician on staff, training on depression
screening, etc.; collaboration indicators may be limited to partnerships with mental health providers;
coordination activities could be focused on depression screening and referrals to mental health treatment;
and the outcome of interest may be whether participants receive at least three weeks of therapy. As
another example, awardees may choose to use the predictor variables described in this section as is, but,
based on findings from Study A, change the outcome variable from service connection to self-advocacy. The
analytic model proposed here is flexible and can be run as long as there is a sufficient sample size, and data
are collected at all three levels of the model (community/LIA/program, home visitor, and family). As
mentioned above, if these conditions are not met, the data can still be analyzed descriptively. Furthermore,
as described later, there are technical assumptions about the data that need to be met to use MLM
appropriately. Awardees without necessary in-house expertise should engage an experienced data analyst
or external evaluation partner to conduct these analyses.
Management information system
Data on the following will be drawn from the home visiting programs’ MIS: family demographic information,
home visiting service utilization (program tenure, dosage, etc.), family receipt of home visitor coordination
activities (e.g., referrals, follow-ups), and family service outcomes (e.g., connected to service, enrolled in
service, received benefit).
Surveys
For the program manager and home visitor surveys, we suggest adapting a survey
j
developed by West et al.
for use in their study of site-level home visiting service coordination (2021),
34
based on the key constructs
identified in Study A. For instance, to modify an existing concept or survey item based on feedback gathered
from participants in Study A, we suggest changing the wording of survey items to reflect this feedback, and
then testing the wording with a small group of staff and families before finalizing the survey item. For a new
concept that is identified in Study A, we recommend drafting survey items that reflect the examples and
participant descriptions from the focus groups. These survey items should also be tested with individuals via
cognitive interviewing to ensure they are capturing the meaning of the concepts identified in Study A.
j
While this survey has not yet been made publicly available; we recommend contacting lead author Allison West at Johns Hopkins
Bloomberg School of Public Health for inquiries related to its use.
A Study Design to Explore Service Coordination in Home Visiting
22
Cognitive testing can be informal and completed in a few hours with a small number of people from your
target audience, or they can be more robust with a formal protocol that dives deeply into question
interpretation. See the resource list at the end of this report for more information on cognitive tests, as well
as resources on the development and administration of surveys.
It is important to note, in the sections that follow, that each category of indicators is assessed at more than
one level of the home visiting system. For instance, coordination activity indicators comprise both (a) home
visitors’ self-reports about how they generally make referrals (Level 2, with home visitor as unit of analysis)
and (b) MIS data about referrals home visitors are making to specific families (Level 1, with family as unit of
analysis). These levels are denoted throughout this section. The examples below are drawn primarily from
the West et al. survey as written; presumably these will change depending on findings from Study A.
Program manager survey
This survey will help awardees understand how program managers perceive coordination efforts within
their program and community and should cover topics related to program characteristics (e.g.,
demographics and background information) and three of the four indicator types:
• Community service indicators, including whether program managers are aware of existing community
service providers and perceive them as accessible to families participating in home visiting.
o Example items include Are the following services available in your community [list service areas, e.g.,
housing, prenatal care, substance use treatment]? What barriers make it hard for families to access
services?
• Implementation system indicators, including whether program managers provide written policies and
procedures for screening and referrals and offer training and support around service coordination.
o Example items include Does your program have job descriptions with clearly defined expectations and
accountability for making referrals? Does your program provide all home visitors with formal trainings
on referral-making?
• Collaboration indicators, including whether program managers have MOUs or other formal agreements
with community service providers, the perceived strength of those relationships, and if the program is
engaged in any cross-sector initiatives (e.g., early childhood coalitions, data sharing).
o An example item is Thinking about the community in which your program operates, to what extent do
you agree that the community has a clearly defined vision and shared goals for service coordination
among early childhood agencies?
• Program characteristics, including program manager race and ethnicity, years of experience in the field,
years of experience in current position, size of home visiting program, years of program operation, and
the type of agency the home visiting program is situated in (e.g., local health department, school system,
community-based organization).
Home visitor survey
This survey will help awardees understand how home visitors perceive coordination efforts within their
program and community, and should cover topics related to home visitor characteristics (e.g., demographics
and background information) and three of the four indicators:
• Implementation system indicators, including whether home visitors are aware of policies and
procedures for screening and referrals, and feel like they have adequate training and supports for
coordination activities.
A Study Design to Explore Service Coordination in Home Visiting
23
o Example items include I feel like I know where to refer families when they need XYZ services; My
supervisor and I talk about service coordination during supervision.
• Collaboration indicators, including whether home visitors have existing relationships with community
service providers and the perceived strength of those relationships.
o Example items include Rate how strong your relationship is with XYZ providers; Rate how easy it is to
get in touch with XYZ providers when you need to.
• Coordination activity indicators, including the types of screening and referral activities home visitors
typically conduct with families, including instrumental supports such as providing help with filling out
forms and making warm hand-offs.
o An example item is Please indicate how often you do the following when making a referral for a family
(list activities, call the provider with families, provide families with information, help families complete
an application)
• Home visitor characteristics, including home visitor race and ethnicity, years of experience in the field,
years of experience in current position, educational background, prior work experience relevant to
service coordination (e.g., past experience as a social worker or case manager), and personal experience
using services in the community.
Family survey
This survey will help awardees understand how participating families perceive coordination efforts within
the home visiting program and community, and should cover topics related to family characteristics (e.g.,
demographics and background information), one of the four indicators, and family outcomes:
• Coordination activity indicators, including whether families engaged in specific referral activities with
their home visitor (e.g., home visitor provided information, helped family complete application, made a
warm hand-off).
o Example items include My home visitor has provided me with information about services in the
community; My home visitor has helped me complete an application for a service in the community.
• Family characteristics, including race and ethnicity, educational background, length of participation in
the program, length of time living in the community, and experience using services in the community.
• Family connection to services, including whether they received the benefit/enrolled in service.
A Study Design to Explore Service Coordination in Home Visiting
24
Table 2. Indicator Category and Key Constructs Mapped onto MLM Levels and Data Sources
Indicator
Category
Example Key Constructs (and Indicators)*
Level of Analysis
Data Source
Family
Home
visitor
Program
MIS
Family
survey
Program
manager
survey
Home
visitor
survey
Service Connection (Outcome Variables)
Family
Connection to
Service
Initial connection to service (e.g., spoke with
someone, scheduled first appointment)
x
x
x
Enrollment/receipt of service (e.g., had first
therapy appointment, received WIC benefit)
x
x
x
Supports and Activities that Influence Service Coordination (Predictor Variables)
Community
Service
Indicators
Availability of community service providers (e.g.,
program knows at least one mental health
provider, pediatric provider, etc.)
x
x
Accessibility of community service providers (e.g.,
barriers to access, ease of access, quality of
programs)
x
x
Coordination among community service providers
(e.g., coalitions, initiatives, coordinated intake)
x
x
Shared infrastructure (e.g., cross-sector/agency
indicators, trainings, competencies)
x
x
Implementation
System
Indicators
LIA/program
policies and
procedures
Supervisor understanding (e.g.,
policies/expectations for
service coordination, hiring
practices)
x
x
Home visitors understanding
(e.g., role clarity)
x
x
A Study Design to Explore Service Coordination in Home Visiting
25
Indicator
Category
Example Key Constructs (and Indicators)*
Level of Analysis
Data Source
Family
Home
visitor
Program
MIS
Family
survey
Program
manager
survey
Home
visitor
survey
Training/
support (e.g.,
formal training
on referral-
making/tracking
and screening,
supervision)
LIA/program provision of
training/support
x
x
Home visitor perceived
adequacy of training/support
x
x
Data systems (e.g., data systems for referral-
tracking, data linkages with other agencies)
x
x
Finance (e.g., dedicated funding for coordination
initiatives, incentives for staff)
x
x
Collaboration
Indicators
Structural indicators (e.g., colocation, MOUs and
formal agreements, designated points of contact)
x
x
x
x
Participation indicators (e.g., program and/or home
visitors sit on other providers’ advisory boards and
vice versa, work as part of coalition or initiative
with other providers)
x
x
x
x
Relational indicators (e.g., close working
relationships with other providers, feels
appreciated by other providers)
x
x
x
x
Coordination
Activity
Indicators
Time spent on
service
coordination
General practice: how much
time home visitors typically
spend on service coordination
activities
x
x
Family perspective: how much
of home visits spent on service
coordination
x
x
A Study Design to Explore Service Coordination in Home Visiting
26
Indicator
Category
Example Key Constructs (and Indicators)*
Level of Analysis
Data Source
Family
Home
visitor
Program
MIS
Family
survey
Program
manager
survey
Home
visitor
survey
Assessing service
needs
General practice: how home
visitors assess families’ service
needs (e.g., screenings, intakes,
follow-ups)
x
x
Family service receipt: were
assessed for their service
needs (e.g., received screen,
intake, etc.)
x
x
x
Referral-making
activities
General practice: how home
visitors typically make
referrals (e.g., information
provision, help complete
applications, warm hand-offs)
x
x
Family service receipt:
received referral activity (e.g.,
home visitor provided
information, helped family
complete application, made a
warm hand-off)
x
x
x
Referral follow-
up activities
General practice: how home
visitors typically follow up on
referrals (e.g., check in with
family, follow-up with
provider, troubleshoot)
x
x
Family service receipt:
received follow-up activities
(e.g., home visitor checked in
with family, followed-up with
provider, helped family
troubleshoot)
x
x
x
A Study Design to Explore Service Coordination in Home Visiting
27
Indicator
Category
Example Key Constructs (and Indicators)*
Level of Analysis
Data Source
Family
Home
visitor
Program
MIS
Family
survey
Program
manager
survey
Home
visitor
survey
Intensity of home visitor referral activities
x
x
x
x
x
Program, Home Visitor, and Family Characteristics
LIA/Program
Characteristics
Program size
x
x
Program tenure (i.e., years in existence)
x
x
Program auspices (e.g., child welfare, community
health)
x
x
Program setting (i.e., rural/urban)
x
x
Home Visitor
Characteristics
Demographic information (e.g., race, education)
x
x
Length of time living in community
x
x
Personal experience using community services
x
x
Program tenure
x
x
Family
Characteristics
Demographic information (e.g., race, education)
x
x
x
Service utilization (e.g., length of enrollment,
service level)
x
x
x
*As noted above, key constructs and indicators used in the final model should reflect awardee interest and findings from Study A. We provide these just as examples to demonstrate how
the multilevel, multi-data source study could be operationalized.
28 A study design exploring service coordination in home visiting
Analysis plan
This section presents a plan for analyzing the data obtained via surveys and MIS data. We provide high-level
guidance; for those unfamiliar with data analysis, we recommend working with your organization’s
evaluation lead and/or contract with an external evaluator. Those individuals can prepare more detailed
analysis plans based on the specific surveys and MIS data used for the study.
Data cleaning, preparation, and organization
MIS data
Query and export required data based on the study time frame. This dataset ultimately needs to be in the
form of one row of data per family. Some of the variables recommended for this analysis may require
working with “long” data, where data from one family are stored in multiple rows. When creating variables
based on, for example, the number of referrals families received in a particular timeframe, it may be easiest
to do the basic data cleaning and derivation in long format before transforming it to wide format (i.e., one
row per family).
Program manager and home visitor surveys
Assuming the survey was administered electronically, export the data into a format that can be read by the
data analysis software you plan to use (e.g., Stata, SAS, R). If data were not collected electronically, manually
input the data into a spreadsheet. Then, review the data for:
• Duplicates (i.e., an individual accidentally submitting two responses)
• Missing data, including ensuring that missing values are coded properly
• Formatting to ensure all variables are amenable to analysis (e.g., that numeric variables are stored in a
numeric format)
• Any signs of data errors (e.g., unreasonable values, contradictory responses, incorrect execution of skip
patterns)
• Any indication of non-response bias (i.e., whether those who completed the survey differed from those
who did not)
Next, create any variables needed for analysis. Begin by running summary statistics and summarizing
themes from all open-ended responses. For example, perhaps you asked about how home visitors typically
make referrals. Home visitors may have responded in multiple ways, such as “electronically via email,” “via
email,” or “I usually send an email to the other provider.” You can combine these similar responses so you
can analyze differences between ways of making referrals later.
For this design, it is very important to ensure all data includes unique identifiers (IDs) that facilitate linking
across the three levels. That is, participants will need to be linked to home visitors, and participants and
home visitors to programs. Each “lower-level” working dataset needs the “upper-level” IDs included within.
This means the family-level dataset must include home visitor IDs for each family (pertaining to their
primary home visitor) and program IDs (pertaining to their primary program). If IDs do not exist for home
visitors or programs (e.g., names are used instead), create IDs and keep a list linking names to IDs in a secure
place. For example, Table 3 shows example data from nine families who were served by five different home
visitors working in three programs.
A Study Design to Explore Service Coordination in Home Visiting
29
Table 3. Example of Nested Data
k
k
This example is adapted from Fauth, R. C., Latimore, A., Kelley, S., Goldberg, J., & Supplee, L. (2020). MIECHV program: Developing
strategies to facilitate Cross-Model Collaboration and Data Sharing (MODS): Data management and analysis resource guide. Rockville, MD:
Health Resources & Services Administration, U.S. Department of Health and Human Services.
Program ID
Home Visitor ID
Family ID
1001
142
1
1001
142
2
2001
251
3
2001
251
4
2001
252
5
2001
252
6
3001
312
7
3001
312
8
3001
314
9
Data analysis
l
l
Analytic plan adapted from Fauth, R. C., Latimore, A., Kelley, S., Goldberg, J., & Supplee, L. (2020). MIECHV program: Developing
strategies to facilitate Cross-Model Collaboration and Data Sharing (MODS): Analytic plans report. Rockville, MD: Health Resources &
Services Administration, U.S. Department of Health and Human Services.
The analysis plan moves from descriptive to multivariable analysis. Beginning with descriptive analysis
provides an opportunity to understand the data and the basic relationships between variables. It is also
critical in helping to inform which final set of variables should be included in the multivariable models.
Descriptive analysis
Once data cleaning is complete, the first step is to run descriptive analysis (i.e., univariate and bivariate) and
create separate summaries of findings from each of your data sources—family survey, home visitor survey,
program manager survey, and MIS data. Univariate statistics include tabulations and summaries to check
central tendency including means, distributions such as ranges and standard deviations, and sample sizes.
Bivariate analyses include correlations, crosstabs, and ANOVAs. These analyses explore the relationships
between variables measured at the same level and at different levels.
Once the four working datasets—family (MIS and survey), home visitor survey, and program manager
survey—have been prepared and cleaned, they can be merged into a multilevel dataset. Before creating the
multilevel dataset, make sure to document a final summary of the sample sizes in each dataset, including the
total number of program units, home visitor units, and families.
Examine how each predictor variable is related to each of the outcomes by asking the following questions of
your data:
• What are the associations between service coordination supports and activities and characteristics of
the community, program, home visitor, and family?
• What are the associations among service coordination supports and activities and the family-level
outcome of interest?
• What are the associations among program, home visitor, and family characteristics, and the family-level
outcome of interest?
A Study Design to Explore Service Coordination in Home Visiting
30
Multivariable multilevel analyses
After gaining a thorough understanding of the data, the next step is to move to the multivariable multilevel
framework to test hypotheses and expand on the bivariate analyses. Here, it is possible to examine multiple
variables measured at different levels simultaneously to understand which variables predict family service
outcomes above and beyond the influence of the others. The model is guided by hypotheses and built piece-
by-piece with increasing complexity. The model starts with an examination of variance at the home visitor-
and program-levels (Step 1), moving on to examine the contribution of family-level predictors (Step 2). The
analysis will help you see whether any of the associations between family-level predictors and outcomes
vary between home visitors and between programs (Step 3). Home visitor-level predictors can then be
added to the model (Steps 4 and 5), then program-level variables (Step 6), and finally any interactions
(including cross-level interactions) of interest to assess moderation (Step 7).
Each of the steps is summarized in Table 4 and text below.
Table 4. Suggested Estimation Strategy
Step
Model
Purpose
1
No predictor variables, random
intercept model
Used to determine whether
outcome varies between home
visitors and between programs
2
Family-level fixed effects
Family-level fixed effects indicate
the association between family-
level predictors and the outcome
3
Model 2 + random slopes for
(selected) family-level predictors
Random slopes reveal if the
association between family-level
predictors and the outcome vary
between home visitors and
between programs
4
(adapted) Model 3 + home visitor-
level fixed effects
Home visitor-level fixed effects
provide information on the
association between home
visitor-level predictors and the
outcome
5
Model 4 + random slopes for
home visitor-level predictors
Random slopes of home visitor-
level variables reveal if the
associations between home
visitor-level predictors and the
outcome vary between programs
6
(adapted) Model 5 + program-
level fixed effects
Program-level fixed effects
indicate the associations between
program-level predictors and the
outcome
7
Model 6 + cross-level interactions
Hypothesis-driven cross-level
interactions indicate whether the
associations between predictors
at one level and the outcome vary
by values on a predictor at
another level
A Study Design to Explore Service Coordination in Home Visiting
31
Step 1
It is useful to start with a random intercept model with no predictors to understand variation in the outcome
of interest across home visitors and across programs. The random intercept model partitions the outcome
variance between levels and is used to calculate the proportion of outcome variance attributable to families,
home visitors, and programs, respectively.
Step 2
Next, family-level variables are entered into the analytic model. The family-level variables are referred to as
“fixed effects” in Step 2, as the association between the variables and the outcome is assumed to be constant
across Level 2 and Level 3 units.
Step 3
Random slopes are added to the model in Step 3. Random slopes enable assessment of whether the
association between a family-level characteristic and the outcome is the same for everyone or if it is
different depending on the family’s home visitor or local program. Step 3 is assessing the presence of
“random effects.” Here the model starts to get more computationally demanding; thus, it is best practice to
test random slopes one at a time based on a pre-determined list of hypotheses, past research, or some a
priori intuition that a family-level characteristic varies depending on home visitor or program.
Step 4
After adapting the model from Step 3 as needed, home visitor-level fixed effects can be added to the model.
Like Step 2, the home visitor-level variables entered in Step 4 are assumed to be constant across Level 3
units.
Step 5
Like Step 3, random slopes for home visitor predictors can be added to the program-level component of the
model to assess whether the association between a home visitor-level predictor and the outcome is the
same or varies across programs.
Step 6
Program-level predictor variables are added to the model.
Step 7
Based on hypotheses, cross-level interactions—interactions between two variables measured at different
levels—are added in this step.
These steps present a very simplified overview of the analytic strategy as a guide to structuring the analysis.
The ultimate specification of models is dependent on hypotheses and the structure of the data.
A Study Design to Explore Service Coordination in Home Visiting
32
Practical Considerations
There are many practical considerations to weigh for the proposed study designs. We have outlined key
considerations below.
Technical skill required: For Study A, the proposed study will require a study team that includes a project
director who is responsible for all aspects of the study design and implementation, and one to two data
collection research support staff and/or data analysts. Data collection staff leading focus groups with
families and home visitors should have experience in conducting focus groups. If necessary, training may
need to be provided to staff. For Study B, the proposed study design will require a study team that includes a
project director who is responsible for all aspects of the study design and implementation and two to three
research staff including a skilled data analyst with considerable experience in multilevel modeling. Both
studies would also benefit from the involvement of other key advisors to advise on focus group and survey
protocol development, sample selection, and interpretation of findings.
Level of effort: For Study A, we have assumed a one-year timeline which consists of approximately a four-
month planning period, a four-month data collection period, and a four-month data analysis period. The
planning period includes the development of the focus group protocols and the identification of the sample
of home visitors and families to participate. We estimate approximately 15 to 60 percent effort for two
research staff, which includes time for reviewing documents, scheduling and conducting focus groups, and
analyzing data.
For Study B, we have assumed an 18-month timeline which consists of a six-month planning period, a four-
month data collection period, and an eight-month data analysis period. The planning period includes the
development of the survey protocols and the identification of the sample to participate in surveys. We
estimate approximately three to four research staff at 20 to 40 percent effort each. In addition to the
project director, the research team should include data collection staff to develop the survey protocol,
program the survey in a web-based software tool (e.g., Qualtrics), and manage data collection; and a data
analyst who will manage and clean survey data and program-level MIS data, as well as develop, program, and
run the multilevel models.
For each study, the planning period may also include Institutional Review Board (IRB) review, depending on
the study’s rationale and methods. In general, studies may require IRB approval if they meet the definition of
research, involve human subjects, include interaction with human subjects, or involve access to identifiable
private information. Research can be defined as a systematic investigation designed to develop or contribute
to generalizable knowledge. Typically, studies designed solely to inform quality improvement efforts do not
need IRB approval. For Study B, it is possible that researchers will have access to identifiable private
information from LIA MIS data which may necessitate an IRB review.
Costs: For both Study A and Study B, our estimated costs include funds for data collection and analysis in
addition to staff time. For Study A, in addition to staff time, there are additional costs for data collection and
analysis. As described earlier, there may be costs for software (e.g., Dedoose or NVivo for qualitative data
analysis), recording devices, and transcription of focus groups, as necessary. We recommend providing
incentives to all families and home visitors who participate in focus groups. For a 90-minute focus group, we
recommend approximately $40 per person. In some cases, home visitors may not be able to accept an
incentive for participating in the focus groups. In this case, it may be advisable to provide a stipend to the
home visiting program.
Our estimated budget for Study B also includes costs associated with data collection and analysis in addition
to staff time. Although there are free statistical software options (e.g., R), states may choose to invest in
other software to administer the web-based survey (e.g., Qualtrics), and to analyze the multilevel data (e.g.,
Mplus, SAS) that will incur a cost. We recommend providing incentives to survey respondents. Depending
on the number of respondents and the length of the survey, you can provide gift cards to all respondents or
A Study Design to Explore Service Coordination in Home Visiting
33
arrange a lottery that respondents are entered into for the chance to win a gift card. Our cost estimates
assume $5 gift cards per survey. As described previously, to implement Study B, LIAs will need to enter data
into a web-based MIS that includes all the relevant data points. While the study presumes that an MIS is
already in place, there may be modifications required to the system, which in turn may necessitate training
and other supports for staff. The estimates here do not include these possible additional costs.
As shown in Figure 4, the estimated cost for Study A ranges from about $85,000 to $212,000 depending on
the specific assumptions used. Estimated costs for Study B range from about $270,000 to $464,000, as
shown in Figure 5. Figures 4 and 5 use a staff salary rate of $130/hour for the project director and
$100/hour for other staff, inclusive of fringe and benefits. Awardees and evaluators will need to adjust the
salary rate to reflect their own pay scales.
Figure 4. Study A Cost Considerations
A Study Design to Explore Service Coordination in Home Visiting
34
Figure 5. Study B Cost Considerations
Ethical considerations: For both Study A and Study B, research staff will need to complete trainings on
privacy and confidentiality during the planning period (or before any data collection). Research staff will be
expected to explain the study to participants, ensure their participation is voluntary, and protect their
confidentiality to the extent possible. If Study A or Study B necessitates IRB review, there may be additional
safeguards in place, like written documentation of consent. Research staff should obtain verbal consent
from all focus group participants before beginning the protocol. The survey should require respondents to
signal their consent (e.g., through a check box) before any survey questions are asked.
Use of Findings
The studies proposed in this report should provide home visiting administrators with a more comprehensive
understanding of what service coordination looks like within their own state and community contexts, and
how those systems and supports could be improved. Specific examples of how the findings can be used
include:
Use the aspects of coordination defined by families and home visitors to inform programmatic decision-
making. As programs continue to look for ways to include family voice and home visitor input in
programmatic decision-making, Study A provides a clear way to do this for decision-making related to
service coordination. For instance, results from Study A could point towards a re-conceptualization of how
LIAs should be conducting screening and assessments, making referrals, and collaborating with community
service providers. This might lead to changes in program operations, home visitor roles, staffing (e.g., a new
hire focused on service coordination), and program expectations. Results might also suggest that LIAs need
A Study Design to Explore Service Coordination in Home Visiting
35
to develop additional tools, resources, and data supports for home visitors to better meet families’ service
needs.
Use results to develop technical assistance and training materials for LIAs across the state. The results
from the multilevel analyses in Study B will provide clear insights into where there are gaps in service
coordination activities and where there are opportunities to build upon. For example, Study B might show
that home visitor coordination activity indicators (e.g., calling providers with families) are associated with
increased connection to services for families. This would point to a clear need to develop or strengthen
existing technical assistance and training materials that highlight the importance of these coordination
activity indicators, provide clear instruction and steps on best practices for doing these activities, and give
home visiting staff the opportunity to practice these activities and build their skills.
Use results to conduct outreach, networking, and systems-building activities in local communities. The
results from both studies should highlight areas where additional effort may be needed across the levels of
the home visiting system. For instance, Study B might show that community service indicators such as
shared goals across providers are an important predictor of connection to services for families. This finding
would suggest that LIAs may need to do more outreach, networking, and systems-building activities with
local community service providers focused on establishing shared goals and identifying ways that
community can best support the shared goals.
Conclusions and Next Steps
This report presents a design for a study to increase the understanding of service coordination and its
relationship to service outcomes for families. To our knowledge, there have been no home visiting
evaluations that have systematically examined the associations between the many indicators of service
coordination identified in the measurement framework for service coordination
35
and service outcomes for
families. Given the limited understanding of this topic, this study uses a two-part design: Study A, a
qualitative study to generate a more comprehensive understanding of how those closest to program
implementation understand service coordination and which activities and supports lead to effective service
coordination and family service connections; and Study B, a quantitative study using data collected on the
co-created predictors and outcomes generated in Study A in a multilevel model to examine which aspects of
coordination appear to most strongly influence family service connections.
As described previously, Study A will provide awardees with family and home visitor driven definitions of
the supports and activities that influence service coordination. Study B will provide awardees with a clear
understanding of the supports and activities as well as the program, family, and implementation
characteristics that lead to better family service outcomes. For further work on this topic, awardees could
consider additional studies that include a case study approach. Case studies could be used to contextualize
and explain interesting or surprising findings from the MLM results in Study B, and would provide a richer
and more comprehensive understanding of how service coordination works in particular settings with
particular families. Researchers should select two to three program sites that meet certain criteria
depending on the findings; for instance, they may want to focus on a program site with strong community
collaborations but low family uptake of services, or a site with home visiting staff dedicated to care
coordination in a rural area with few to no available or accessible services. Other possible studies could
include a community-based study to understand what is needed to strengthen coordination from the
perspective of external partners, and a cost study that could explore what additional resources are needed
for the community to develop and strengthen supports and activities that influence service coordination.
For awardees interested in pursuing the studies described in this design report, we encourage them to
review the practical considerations and adapt the study design to their aims and resources, as needed. We
also encourage readers to review the other study design reports that accompany this resource. This suite of
A Study Design to Explore Service Coordination in Home Visiting
36
study designs covers a variety of topics that reflect awardee interests, uses different designs, and varies in
terms of time and resources required.
This document was prepared for the U.S. Department of Health and Human Services (HHS), Health
Resources and Services Administration (HRSA), by Child Trends in partnership with James Bell Associates,
under HRSA contract number 75R60219D00026. The views expressed do not reflect the position of HRSA
or HHS. Authors for the report are Jessica Goldberg, Sarah Crowne, Rebecca Fauth, and Kara Ulmen.
A Study Design to Explore Service Coordination in Home Visiting
37
Resources
Survey development
Fink, A. (2003). The survey handbook, 2
nd
edition. Thousand Oaks, CA: Sage.
Organisation for Economic Cooperation and Development (2012). Good practices in survey design step-by-
step. In Measuring regulatory performance: A practitioner's guide to perception surveys. Paris: OECD Publishing.
https://doi.org/10.1787/9789264167179-6-en
Pew Research Center (2021). Writing survey questions. https://www.pewresearch.org/our-methods/u-s-
surveys/writing-survey-questions/
Cognitive testing
Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research,
12(3), 229-238. https://doi.org/10.1023/A:1023254226592
Analysis
Browne, W.J., Lahi, M.G., & Parker, R.M. (2009). A guide to sample size calculations for random effect models via
simulation and the MLPowSim software package. Bristol, United Kingdom: University of Bristol.
http://www.bristol.ac.uk/media-library/sites/cmm/migrated/documents/mlpowsim-manual.pdf
Flick, U. (2014). The SAGE handbook of qualitative data analysis. London: Sage.
https://dx.doi.org/10.4135/9781446282243
Pazzaglia, A. M., Stafford, E. T., & Rodriguez, S. M. (2016). Survey methods for educators: Analysis and reporting
of survey data (part 3 of 3) (REL 2016–164). Washington, DC: U.S. Department of Education, Institute of
Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional
Educational Laboratory Northeast & Islands.
Scherbaum, C.A., & Pesner, E. (2019). Power analysis for multilevel research. In S.E. Humphrey & J.M.
LeBreton (Eds.), The handbook of multilevel theory, measurement, and analysis (pp. 329–352). American
Psychological Association. doi: 10.1037/0000115-015
University of Bristol (2021). MLPowSim. http://www.bristol.ac.uk/cmm/software/mlpowsim/
A Study Design to Explore Service Coordination in Home Visiting
38
References
1
Azzi-Lessing, L. (2013). Serving highly vulnerable families in home-visitation programs. Infant Mental Health
Journal, 34(5), 376-390. doi: 10.1002/imhj.21399
2
Green, B.L. (2019). Understanding quality in home visiting—how far have we come and where do we need
to go? Infant Mental Health Journal, 40(3), 395-400. doi:10.1002/imhj.21778
3
West, A., Duggan, A. K., Gruss, K., & Minkovitz, C. S. (2018). Creating a measurement framework for service
coordination in maternal and early childhood home visiting: An evidence-informed, expert process. Children
and Youth Services Review, 89, 289-297.
4
Duggan, A., Portilla, X. A., Filene, J. H., Crowne, S. S., Hill, C. J., Lee, H., and Knox, V. (2018). Implementation of
evidence-based early childhood home visiting: Results from the Mother and Infant Home Visiting Program
Evaluation. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and
Families, U.S. Department of Health and Human Services.
5
Duggan, A., Portilla, X. A., Filene, J. H., Crowne, S. S., Hill, C. J., Lee, H., and Knox, V. (2018). Implementation of
evidence-based early childhood home visiting: Results from the Mother and Infant Home Visiting Program
Evaluation. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and
Families, U.S. Department of Health and Human Services.
6
Goldberg, J., Winestone, J.G., Fauth, R., Colón, M., Mingo, M.V. (2018) Getting to the warm hand-off: A
study of home visitor referral activities. Maternal and Child Health Journal, 22(1), 22-32.doi:10.1007/s10995-
018-2529-7
7
Duggan, A., Portilla, X. A., Filene, J. H., Crowne, S. S., Hill, C. J., Lee, H., and Knox, V. (2018). Implementation of
evidence-based early childhood home visiting: Results from the Mother and Infant Home Visiting Program
Evaluation. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and
Families, U.S. Department of Health and Human Services.
8
Duggan, A., Portilla, X. A., Filene, J. H., Crowne, S. S., Hill, C. J., Lee, H., and Knox, V. (2018). Implementation of
evidence-based early childhood home visiting: Results from the Mother and Infant Home Visiting Program
Evaluation. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and
Families, U.S. Department of Health and Human Services.
9
West, A., Duggan, A. K., Gruss, K., & Minkovitz, C. S. (2018). Creating a measurement framework for service
coordination in maternal and early childhood home visiting: An evidence-informed, expert process. Children
and Youth Services Review, 89, 289-297.
10
West, A., Duggan, A. K., Gruss, K., & Minkovitz, C. S. (2020). The role of state context in promoting service
coordination in Maternal, Infant, and Early Childhood Home Visiting programs. Journal of Public Health
Management and Practice, 26(1), E9–E18. doi:10.1097/PHH.0000000000000907
11
West, A., Duggan, A., Gruss, K., & Minkovitz, C. (2021). Service coordination to address maternal mental
health, partner violence, and substance use: Findings from a national survey of home visiting
programs. Prevention Science, 22(5), 633–644. doi:10.1007/s11121-021-01232-9
A Study Design to Explore Service Coordination in Home Visiting
39
12
Goldberg, J., Winestone, J.G., Fauth, R., Colón, M., Mingo, M.V. (2018) Getting to the warm hand-off: A
study of home visitor referral activities. Maternal and Child Health Journal, 22(1), 22-32.
doi:10.1007/s10995-018-2529-7
13
Lowell, D. I., Carter, A. S., Godoy, L., Paulicin, B., & Briggs-Gowan, M. J. (2011). A randomized controlled
trial of Child FIRST: A comprehensive home-based intervention translating research into early childhood
practice. Child Development, 82, 193–208.
14
Dodge, K. A., Goodman, W. B., Murphy, R. A., O’Donnell, K., Sato, J., & Guptill, S. (2014). Implementation
and randomized controlled trial evaluation of universal postnatal nurse home visiting. American Journal of
Public Health, 104(S1), S136–S143.
15
Silovsky, J. F., Bard, D., Chaffin, M., Hecht, D., Burris, L., Owora, A., et al. (2011). Prevention of child
maltreatment in high-risk rural families: A randomized clinical trial with child welfare outcomes. Children and
Youth Services Review, 33(8), 1435–1444.
16
Green, B. L., Sanders, M. B., & Tarte, J. (2017). Using administrative data to evaluate the effectiveness of
the Healthy Families Oregon home visiting program: 2-year impacts on child maltreatment & service
utilization. Children and Youth Services Review, 75, 77–86.
17
Duggan, A., Fuddy, L., Burrell, L., Higman, S. M., McFarlane, E., Windham, A., & Sia, C. (2004). Randomized
trial of a statewide home visiting program to prevent child abuse: Impact in reducing parental risk factors.
Child Abuse & Neglect, 28(6), 623–643.
18
Goldberg, J., Winestone, J.G., Fauth, R., Colón, M., Mingo, M.V. (2018) Getting to the warm hand-off: A
study of home visitor referral activities. Maternal and Child Health Journal, 22(1), 22-32.
doi:10.1007/s10995-018-2529-7
19
Duggan, A., Portilla, X. A., Filene, J. H., Crowne, S. S., Hill, C. J., Lee, H., and Knox, V. (2018). Implementation
of evidence-based early childhood home visiting: Results from the Mother and Infant Home Visiting Program
evaluation. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and
Families, U.S. Department of Health and Human Services.
20
West, A., Duggan, A. K., Gruss, K., & Minkovitz, C. S. (2018). Creating a measurement framework for
service coordination in maternal and early childhood home visiting: An evidence-informed, expert
process. Children and Youth Services Review, 89, 289-297.
21
West, A., Duggan, A., Gruss, K., & Minkovitz, C. (2021). Service coordination to address maternal mental
health, partner violence, and substance use: Findings from a national survey of home visiting
programs. Prevention Science, 22(5), 633–644. doi:10.1007/s11121-021-01232-9
22
Goldberg, J., Winestone, J.G., Fauth, R., Colón, M., Mingo, M.V. (2018) Getting to the warm hand-off: A
study of home visitor referral activities. Maternal and Child Health Journal, 22(1), 22-32.doi:10.1007/s10995-
018-2529-7
23
Krueger, R.A. & Casey, M.A. (2008) Focus groups: A practical guide for applied research. 4th edition. New
York: SAGE.
24
Goldberg, J. & Litovich, M. (2018). Massachusetts Maternal, Infant, and Early Childhood Home Visiting (MA
MIECHV) formula grant evaluation: Final report to the Massachusetts Department of Public Health. Medford, MA:
Tufts Interdisciplinary Evaluation Research; Shrewsbury, MA: UMass Donahue Institute.
A Study Design to Explore Service Coordination in Home Visiting
40
25
West, A., Duggan, A., Gruss, K., & Minkovitz, C. (2021). Service coordination to address maternal mental
health, partner violence, and substance use: Findings from a national survey of home visiting
programs. Prevention Science, 22(5), 633–644. doi: 10.1007/s11121-021-01232-9
26
Goldberg, J., Winestone, J.G., Fauth, R., Colón, M., Mingo, M.V. (2018) Getting to the warm hand-off: A
study of home visitor referral activities. Maternal and Child Health Journal, 22(1), 22-32.
doi:10.1007/s10995-018-2529-7
27
West, A., Duggan, A. K., Gruss, K., & Minkovitz, C. S. (2018). Creating a measurement framework for
service coordination in maternal and early childhood home visiting: An evidence-informed, expert
process. Children and Youth Services Review, 89, 289-297.
28
Rosinsky, K., Madill, R., Bashara, S., Supplee, L., Shaw, S., Stearns, R., Li, W., Gutowski, T., Cantrell, E. (2019).
Assessment and mapping of community connections in home visiting: Final report. Washington, DC: Office
of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health
and Human Services.
29
Scherbaum, C.A., & Pesner, E. (2019). Power analysis for multilevel research. In S.E. Humphrey & J.M.
LeBreton (Eds.), The handbook of multilevel theory, measurement, and analysis (pp. 329–352). American
Psychological Association. doi: 10.1037/0000115-015
30
Browne, W.J., Lahi, M.G., & Parker, R.M. (2009). A guide to sample size calculations for random effect models
via simulation and the MLPowSim software package. Bristol, United Kingdom: University of Bristol.
http://www.bristol.ac.uk/media-library/sites/cmm/migrated/documents/mlpowsim-manual.pdf
31
University of Bristol (2021). MLPowSim. http://www.bristol.ac.uk/cmm/software/mlpowsim/
32
Browne, W.J., Lahi, M.G., & Parker, R.M. (2009). A guide to sample size calculations for random effect models
via simulation and the MLPowSim software package. Bristol, United Kingdom: University of Bristol.
http://www.bristol.ac.uk/media-library/sites/cmm/migrated/documents/mlpowsim-manual.pdf
33
Browne, W.J., Lahi, M.G., & Parker, R.M. (2009). A guide to sample size calculations for random effect models
via simulation and the MLPowSim software package. Bristol, United Kingdom: University of Bristol.
http://www.bristol.ac.uk/media-library/sites/cmm/migrated/documents/mlpowsim-manual.pdf
34
West, A., Duggan, A., Gruss, K., & Minkovitz, C. (2021). Service coordination to address maternal mental
health, partner violence, and substance use: Findings from a national survey of home visiting
programs. Prevention Science, 22(5), 633–644. doi: 10.1007/s11121-021-01232-9
35
West, A., Duggan, A. K., Gruss, K., & Minkovitz, C. S. (2018). Creating a measurement framework for
service coordination in maternal and early childhood home visiting: An evidence-informed, expert
process. Children and Youth Services Review, 89, 289-297.