Copyright © 2016 Making Connections - NORC at the University of Chicago. All Rights Reserved.
Making Connections News
NORC is pleased to provide an update regarding the recent activities of the Making Connections Research Scholars.
The Making Connections Research Scholars joined colleagues from the Urban Institute, NORC at the University of Chicago, Chapin Hall Center for Children, and Case Western Reserve University to organize two sessions at the 2015 meeting of the Urban Affairs Association in Miami, FL. The first panel, titled “Using Making Connections Data to Explore Influences, Experiences and Perceptions of Urban Life in Disadvantaged Neighborhoods,” featured research by Megan Gilster, Anita Zuberi, Amanda Roy, Jessica Burdick-Will, and Carlos Siordia. Daniel Brisson, Claudia Coulton, Robert Goerge, Brett Theodos, and Emily Zimmerman presented in the second session, a culloquoy devoted to methodological challenges and associated research findings.
Listed below are the names of the Research Scholars followed by working titles for their research.
- Daniel Brisson: "Neighborhood Social Cohesion for Child Health and Well-Being"
- Julia Burdick-Will: "Decoupling Schools and Neighborhoods: Using Making Connections to Understand How Opting Out of Local Schools Shapes Neighborhood Perceptions"
- Megan Gilster: "Community Social Organization and Residential Mobility: Exploring Resident Participation and Child Health in Making Connections Neighborhoods"
- Amanda Roy: "Modeling the complexities of organizational resource use: A mediating pathway between neighborhood disadvantage and child health and behavior problems"
- Carlos Siordia: "Defining "Neighborhood" Geographical Boundaries through Empirical Methods"
- Emily Zimmerman: "Educational and Health Outcomes: Neighborhood and Individual Mediators"
- Anita Zuberi: "Low-Income Neighborhoods and Health: Assessing the Influence of Physical and Social Neighborhood Conditions on Child Health"
What is Making Connections?
The Making Connections initiative, funded by the Annie E. Casey Foundation (AECF), is a collaboration of local organizations and residents that seeks to improve outcomes for disadvantaged children by strengthening their families, improving their neighborhoods, and raising the quality of local services. This initiative includes a diverse set of places in terms of race, ethnicity, immigrant populations, physical and economic conditions, yet all are disadvantaged relative to their surrounding metropolitan areas. Information about the Making Connections initiative may be found on the AECF website.
Frequently Asked Questions
About the Survey
- How were the various sites and neighborhoods selected?
In 1999, the Annie E. Casey Foundation identified 22 cities to be part of an exploratory phase for a new neighborhood transformation/family development initiative. In each site, local teams including members of the Casey staff, national and local consultants, representatives from community foundations and neighborhood associations, and city agencies were formed. Three years later, the Foundation selected ten communities for a cross-site investigation in the form of a household survey. These sites were chosen because they had community support organizations that 1) were engaged in community outreach that was consistent with the Foundation's mission to support families with children and 2) could facilitate data collection on the ground. AECF "was purposeful in selecting neighborhoods that demonstrated characteristics such as institutional support that would increase the probability of program success" (Brisson and Usher 2005, 650). The Foundation sought help from the local partners define the geographic boundaries of the survey sites. The resulting boundaries were often nuanced based on local interests. They are not synonymous with Census-defined geographies. For example, the survey neighborhood in San Antonio was drawn to include residents mainly in the Westside community but also some from within the Edgewood school district (Bachtell 2012).
- How big are the Making Connections Survey neighborhoods?
The median size of the ten Making Connections Survey neighborhoods as of the baseline (2002-2004) is 4.92 square miles and includes a population of 30,598 people (Census 2000). See appendix A.1 in Coulton et al 2009.
- What was the subject inclusion criteria?
- 18 years old or older
- Living in a selected household
- How was the data collected?
At baseline almost all interviews were conducted in-person. For both Wave 2 and Wave 3 NORC interviewers contacted respondents and conducted interviews by telephone and in-person; roughly half of the interviews were conducted by telephone and half in-person. The in-person interviews were conducted in the respondent's home or, occasionally, outside the respondent's home in a location that allowed enough privacy to conduct an interview (for example, the respondent's front porch, car, workplace, etc.). The interviewer first enumerated all individuals living in the household and then scientifically selected one adult (age 18 or older) to act as the respondent and answer the survey questions. The interviewer then read questions from a series of paper-and-pencil questionnaires and recorded the respondent's answers. Interviews typically lasted 50 minutes.
- Who made up the interviewing team?
Roughly 30 NORC field interviewers per site were recruited for each wave of data collection. Preference was given to those who lived in or near the target survey neighborhoods and/or had conducted interviews for a previous wave of the survey. Interviewers were supervised by an NORC field project manager. There were two field managers per site. The field managers provided on-the-ground guidance during the first two weeks of data collection and maintained regular contact with interviewers by telephone throughout the field period.
- What was the process for gaining informed consent?
After enumerating the members of the household and scientifically selecting the respondent, the interviewer read the following Informed Consent statement to the respondent:
"This interview takes about 50 minutes, but the time may be shorter depending on how many questions you answer. I will be asking about your neighborhood, such as its services and amenities, community organizations, volunteerism, and your household. If you have children living with you, there will also be a section about each of them. Your participation in this survey is completely voluntary. You are free to stop the interview at any time. If there is a question you would rather not answer, I will skip it. Some of these questions might not apply to you, but I need to ask each question. All information you give is strictly confidential, and your name is never associated with the results. Findings of the research will be reported only in summary form, so that no individuals can ever be identified. I think you will find this an interesting and worthwhile experience. May we begin the interview now?"
Verbal consent from the respondent was required to proceed.
- Were respondents paid for participating in the survey?
Yes, respondents were paid $20 via money order as a thank-you for participating in the interview and received this amount even if they chose to skip one or more questions or terminate the interview early.
- What about non-English speaking households? How did you interview them?
Translated materials and bilingual interviewers were employed in sites in which 10% or more of the city population spoke a language other than English, based on recent Census data. Eight of the ten sites included Spanish components (all but Milwaukee and Louisville). Vietnamese was used in 3 sites: Denver, White Center (Seattle), and Oakland. Oakland also supported Mandarin and Cantonese dialects. All interviewers, monolingual or multilingual, signed pledges of confidentiality as a part of the hiring process.
If the interviewer discovered that the selected respondent did not speak and/or understand neither English nor the other language(s) supported in that site (Spanish, Vietnamese, etc.) the interviewer was required to first obtain permission from the respondent to conduct the interview with the help of an interpreter of his or her choosing. The interpreter was typically a trusted family member, neighbor, or friend. The interviewer asked the informant to explain this process to the selected respondent and ask for his/her permission to proceed: "Before we begin I would like to ask the respondent's permission to have you translate the interview. Please translate the following:
You have been selected to be the respondent. We do not have a [LANGUAGE] language version of the questions I need to ask you. I must interview the person scientifically selected to represent this household, and that person is you. The person who translates the questions and your responses should be a trusted family member or friend who will keep confidential the questions I ask and the answers you provide. Who would you like to assist me by translating the questions for you and translating your answers for me? To thank (him/her) for (his/her) time, I will give [INTERPRETER NAME] $20. Do I have your permission to conduct the interview with the help of [INTERPRETER]?"
- The survey contains many questions about children. Did you talk directly with children?
No, the survey administration did not involve collecting data directly from children. NORC interviewers asked all questions, including those about the children living in the household, of the respondent. Only adults age 18 and over were eligible to act as the respondent. We made one exception for "minor parents." If the primary caregiver of the selected focal child happened to be under age 18, that parent could act as the respondent provided that we secured both assent from the minor respondent and permission from his or her parent. These cases were very rare. At most we had 20 minor parent respondents out of 28,000 interviews.
- What were the attrition rates between waves?
- 5,252 of the households interviewed in wave 1 remained the same household and in-scope at wave 2. 4,093 of the 5,252 were interviewed again in wave 2. This yields a 22.1% attrition rate between waves 1 and 2 (unweighted).
- 4,190 of the households interviewed in wave 2 remained the same household and in-scope at the wave 3. 3,515 of the 4,190 were interviewed again in wave 3. This yields a 16.1% attrition rate between waves 2 and 3 (unweighted).
Note that these rates reflect our best intelligence as to the whereabouts of previous wave focal children. "Mover" cases were added to the sample in waves 2 and 3 if we discovered that the child previously selected as the focal child had moved to a new location. We cannot account for attrition among households in which the location of the previous wave focal child could not be determined due to refusals, language barriers, etc. We also did not attempt to interview any households without children that moved between waves.
- Were respondents informed of the survey results?
With some guidance from Annie E. Casey Foundation and Urban Institute staff, the local site partners were responsible for communicating the survey findings to community members. For example, most sites prepared a one or two-page summary document that NORC included in a postcard mailing to respondents in between waves 2 and 3. Some sites have posted select findings on their websites, as shown here at the Making Connections San Antonio site: http://www.mc-sa.org/.
- Do you have any additional language about how specifically data collection materials were returned/kept/stored?
During the field period, hardcopy materials containing confidential information were returned to NORC facilities via secure UPS mail within one week of the interview date. They were kept in locked storage and access was limited to a small number of approved project staff members for receipt, edit, and data entry processes.
About the Data
- Do you have any language about NORC's data security procedures that I can include in my IRB application?
The Making Connections Survey data are and will remain stored on secure servers at NORC. Access to the data is provided through NORC's Data Enclave, a remote platform that allows secure access to sensitive data. Users must connect to the Enclave using a Citrix system that requires a password and authorization to view the survey data. Data users cannot move files inside or outside of the Enclave without review approval by NORC Data Enclave managers.
The NORC Data Enclave operates under a NIST-approved System Certification and Accreditation (C&A) package, including an approved IT Security Plan, Data Protection Plan, and a System Certification Test Plan, as outlined in DOC IT Security Program Policy, Section 6.5.2. The NORC's Data Enclave IT Security Plan is fully compliant with the Federal Information Security Management Act (FISMA), provisions of mandatory Federal Information Processing Standards (FIPS), and meets all of NIST's IT, data, system, and physical security requirements (http://www.dataenclave.org/index.php/security).
- Do you have any language about potential risks that I can include in my IRB application?
Families could experience adverse consequences if an interviewer or data user disclosed information about an interviewee to a third party or if an interviewer did not follow the data security plan. To mitigate these risks, Interviewers sign pledges of confidentiality as a part of the hiring process and confidentiality is discussed in a stand-alone module during the Making Connections substantive training. Data users are required to sign a non-disclosure agreement indicating that they will act in accordance with NORC's policies for ethical conduct.
- My IRB wants to know if the data that I will be using have been de-identified. Is that the case?
Yes. The data sets inside the Data Enclave (the ones available to approved external researchers) have been stripped of the following information:
- Names of individuals
- Mailing address
- Phone numbers
NORC maintains this information in controlled project folders on our secure servers, outside the Data Enclave, for the purpose of following up with respondents. External researchers do not and will not have access to the folders nor this information. Subjects could potentially still be identified if a given data user had intimate familiarity with the survey neighborhoods and tried to isolate a family using a combination of variables that are included in the Data Enclave datasets. External researchers are required to sign a non-disclosure agreement saying that they will not use the data for this purpose and will act in accordance with NORC's policies for ethical conduct.
- Is there a fee for obtaining the data?
There are costs associated with getting access to the data, but AECF has been graciously absorbing the fees for data users. Currently, the only cost to you would be your time.
- What is the rationale for applying weights to the Making Connections Survey data?
Weighting is necessary to account for different selection probabilities by community/site (i.e. Denver, Des Moines, etc.), neighborhood, and sampling frame, and also to adjust for nonresponse. Different sampling rates were applied within neighborhoods and within communities. For example, our local partners in Denver wanted to complete the same number of interviews in each of the surveyed neighborhoods even though the neighborhoods were of very different sizes. If the weights are not applied, there is a risk of bias in the results.
About Data Workshops
- Do workshop participants receive copies of the Making Connections Survey data sets?
Participants have the opportunity to explore sample versions of the survey datasets during the workshop. To gain access to the full confidential files, you would need to complete the application process outlined at http://mcstudy.norc.org/data-access/.
The Making Connections Survey is a set of longitudinal data collection activities conducted over a ten year period, in ten US cities at two points in time, and in seven of the ten cities at three points in time. The Making Connections survey is designed to measure how neighborhood change affects the well-being of children and families.