April 9, 1999
MEMORANDUM TO SPD Steering Committee
From:
Jenny Hess
Center for Survey Methods Research/SRD
Subject:
Results of cognitive testing on the Residential History Module for
Survey of Program Dynamics 2000
Enclosed are two reports on results of the cognitive testing conducted on the residential history questions for the Survey of Program Dynamics 2000. Dr. Robert Belli (Institute for Social Research, University of Michigan) prepared the first report under contract with CSMR. I prepared the second report. Both reports detail the problems found during cognitive testing. Dr. Belli's report provides general recommendations for revising the questions at the end of the report. In addition to the two reports, there are several attachments. Attachment A is a copy of the questionnaire tested and interviewing protocol. Attachment B is the residential history calendar that was used during the cognitive testing. Attachment C details respondent characteristics for the interviews I conducted. Attachment D contains item-by-item recommendations for revising the questionnaire and calendar as well as issues raised during cognitive testing that need to be resolved. Attachment E is a draft questionnaire based on the recommendations contained in Attachment D and the Belli report. Note that Attachment E does not address all the issues raised in the reports, but is provided in order to give readers the general flow of the recommended questionnaire.
Our basic recommendation is to allow interviewers to use a more flexible interviewing technique to collect the information on separations from biological mothers and fathers and periods of living with other adults. We propose that a flexible interviewing technique be used to fill out the residential history calendar completely before inputting data into the automated instrument. (All data will be input in the automated instrument during the course of the interview.) A flexible technique will allow interviewers and respondents to use the residential history calendar to its fullest extent and not force respondents to proceed through the interview in a linear manner. Once the calendar has been filled out, the interviewer will input the data into the automated instrument and ask all appropriate follow-up questions. If this technique is not adopted, the results from the cognitive interviews suggest that the sponsor will need to consider one of two options, either reduce the scope of content collected and focus only on a few critical residential history elements, or drastically increase the list of scripted questions. We favor using the flexible interviewing technique to fill out the calendar because we believe it will improve data quality and accommodate the scope of content desired.
Please note that Dr. Belli and I conducted our interviews using different methodologies. For most of his interviews, Dr.Belli did not use the structured protocol I had developed because he found it difficult to use. Instead he relied more on a flexible interviewing technique and allowed respondents to "tell their stories." Since our methods differed substantially, I asked him to include a description of how he conducted his interviews in his report ("methodology section"). I used the protocol included in Attachment A with an emphasis are obtaining respondents' understanding of the questions. I did not provide respondents with additional information beyond what is included in the questions, but rather probed them for their understanding of the questions tested. If you would like additional information regarding differences in our interviewing techniques, please let me know.
Please contact me on 457-4968 if you have any questions or comments on the report or recommendations.
To: Jenny Hess, Bureau of the Census
From: Bob Belli, University of Michigan
Date: 3/23/99
Report of Cognitive Interviews for "Event History Calendar Method for Improving Retrospective Recall of Survey Questions" project, using SPD 2000 interviewing protocol.
I. Methodology
Robert Belli conducted six concurrent cognitive think-aloud interviews during February and March of 1999. Respondents were screened for having had minor children who stopped or started living with them. In all interviews, the household roster information was collected following the scripts in the protocols. This included asking questions about eligible children both within and outside the respondents present household, and asking for eligible children of all household members aged 15 and older. In no case was there a situation in which a household member other than the respondent had an eligible child living outside the household.
Table 1: Respondents and Eligible Children
| Gender | Birth date |
Race | Eligible Children | |||
| Sex | Relation to R | In R's house? |
Birth date |
|||
| 1. Female | 3/8/67 | African American | 1.F 2.M 3.F |
bio child bio child bio child |
no no yes |
2/26/85 2/9/88 1/17/98 |
| 2. Female | 7/24/61 | white | 1.M 2.F |
adopted child adopted child |
yes yes |
1/31/77 3/10/79 |
| 3. Female | 12/24/55 | white | 1.M 2. F 3. F |
bio child bio child grandchild |
no yes yes |
7/9/78 4/2/80 7/16/98 |
| 4. Female | 2/16/54 | white | 1. M 2. M 3. F 4. M |
bio child bio child bio child bio child |
no no no no |
7/3/78 12/28/88 7/9/90 5/31/92 |
| 5. Female | 3/16/67 | white | 1. F | guardianship | no | x/x/80 |
| 6. Female | 10/13/53 | Native American | 1. M | bio child | no | 5/19/77 |
Note: x means that this value was not known by the respondent
The methodology for conducting the interviews was mixed. The first four interviews followed the screener questions as written. With the last two interviews, the screener questions (SCRIA - SCR1E) were asked only to the extent that eligible children were identified. In all of the interviews except the third one, the calendar information was collected flexibly with respondents, that is, the scripted questions MOM1 to OTHAD5 were not asked exactly as written, nor in the prescribed order. Instead, the calendar, used interactively with the respondent, was used as a guide to collect the required information. Interestingly, an attempt was made to follow the scripted question sequence of MOM1 to OTHAD5 during the fourth interview, but R4 was too impatient and preferred a more interactive style. In all interviews, while introducing the calendar with INTROCAL, the respondent was shown the respective parts of the calendar as INTROCAL was read, which implicitly invited the respondent to work with the interviewer in filling out the required information. Since the calendar was often dealt with in a flexible manner, the respondent often found themselves telling narratives about their experiences, which were flexibly probed for purposes of recording information as specified by the calendar.
In all interviews, respondents were asked the retrospective probes asking about hypothetical move vignettes, and the debriefing questions concerning the calendar. Additionally, a flashcard was constructed for OTHAD5 and used in all interviews, as an aide for respondents to answer this question. It is recommended that a flashcard be used for this question.
Interviews were audiotaped, and respondents were paid $25 for their participation.
Table 2: Calendar information
| R# | Child # | Living Apart from Biological | Living With | |
| Mother | Father | |||
| 1 | 1 & 2 3 |
x/95 - now | x/85 - now 1/98 - now |
grandmother: x/95 - now grandfather: x/85 - now aunt: x/85 - now |
| 2 | 1 2 |
6/91 - 7/92 8/92 - 3/97 |
6/91 - 10/91 8/92 - 1/95 |
foreign student: 6/91 - 10/91 stepfather: 10/83 - 9/85 nonmarital partner: 6/87 - 10/92 stepfather: 10/83 - 9/85 nonmarital partner: 6/87 - 10/92 stepmother: 10/92 - 9/93 adoptive mother: 9/93 - 3/97 |
| 3 | 1 2 3 |
8/91 - 7/96 8/91 - 3/92 |
11/89 - 8/91 11/89 - 8/91 3/92 - 4/98 7/98 - now |
nonmarital partner: 8/91 - 7/96 nonmarital partner: 8/91 - 3/92 grandmother: 7/98 - now housemate: 8/98 - now housemate: 11/98 - now |
| 4 | 1 2, 3 & 4 |
7/78 - 7/96 12/97 - now |
7/78 - 7/96 12/97 - now |
unknown after adopted: 7/78 - 7/96 foster care: 12/97 - now |
| 5 | 1 | 5/96 - now | 5/96 - now | biological sister: 5/96 - 8/96 brother of sister-in-law: 8/96- 12/96 biological brother: 1/97 - 5/97 guardian: 6/97 - 9/97 biological brother: 9/97 - now housemate: 1/97 - 5/97 housemate: 6/97 - 9/97 |
| 6 | 1 | 11/94 - 5/95 | ||
Note: x means that this value was not known by the respondent
II. Observed Problems
1. CKMF
Having a show card would be helpful as R3 considered a nonmarital partner as a stepmother when read the stepmother category. I also found this section particularly time-consuming -- there were three children to deal with, and going through each of these categories for each child involved considerable looping.
For R5, the child lived with many adult siblings, who served as primary caretakers. Should SCR1C (7) include older siblings? This may be important with those groups who rely more often on extended family relationships.
Both R3 and R5 had reported "housemates" as "other adults." Does the population division want to include these people?
For the most part, respondents did not have difficulty with interpretation of key concepts.
"Biological children" meant:
R2: "children who were conceived by me"
R3: "who I birthed"
R4: "constitutes that child's relationship to parent is by birth"
R5: "I gave birth to them, or if I were the genetic mother"
R6: "children that I physically had and carried around for 9 months"
R2: "marital partner of natural parent"
R2: "room and board, but should mean nurture"
R3: "shelter, food, care"
R5: "providing shelter, food, guidance, keeping an eye on her, making sure child goes to school"
"If a child lived with his mother during the school year and his father for three months during the summer, would you consider that a move?" 5 yes, 1 dk
"If a child lived with her grandmother during the week to attend school, and with her parents on the weekends, would you consider that a move?" 1 yes, 5 no
"If a child spent every other weekend with his father and the rest of the time with his mother, would you consider that a move?" 1 yes, 5 no
"If a child was at boarding school during the school year and at home during the summer, would you consider that a move?" 6 yes
"If a child moved with his family from one apartment to another in the same building, would you consider that a move?" 6 yes
"If a child spent two months during the summer with his father, and the rest of the time with his mother, would you consider that a move?" 4 yes, 2 no
Respondent debriefing questions
1. We used this form (the residential history calendar) to record the your (child/children's) living situation since birth. What did you think about this form?
R2: "holy smokes, should be 3x as big--format is good--color coded in some way instead of variations of grey"
R3: "pretty self-explanatory and made it easier to figure out what was going on in each of those years "
R4: "ages of different children when others were born was neat."
R5: "A little difficult to read -- don't understand numbers 1-15"
R6: [didn't use form enough to know]
R2: "yes, and still didn't do well (respondent was able to provide the months of transitions)"
R3: y
R4: "not sure"
R5: "no -- did math backwards"
R6: [didn't use form enough to know]
R2: y
R3: y
R4: "not sure"
R5: "only with respect to determining age in years"
R6: [didn't use form enough to know]
R2: y
R3: y
R4: "not sure"
R5: "yes -- in the same respect -- and writing down different adults was helpful -- and helpful when pointing to calendar points"
R6: [didn't use form enough to know]
R2: "yes--need to reduce all of those lines of numbers, need to reduce the thickness of lines "
R3: "y --do not make border as bold"
R4: "not sure"
R5: "A little bit -- other people would think so too"
R6: [didn't use form enough to know]
R2: "yes, grade levels not as helpful"
R3: "yes. did talk about first grade
R4: "not sure"
R5: "Yes -- ages and year were the most helpful"
R6: "the age/grade tie was confusing"
1. All respondents, except R1, were able to remember the months of transitional events. Interviewers should be trained as to what to do for respondents who are unable to remember the months of transitions.
2. With R2, there was an extended period in which there was a joint custody following a divorce. During this time, the children were considered to be living with both biological parents, although the biological parents were not living together.
3. R2 served in three roles -- as a nonmarital partner, as a stepparent, and as an adoptive mother, in living with one of the children. With the other child, although she was a stepparent and adoptive parent, she did not live with this other child in these roles. At issue is how to handle these people. I had filled out three other adult lines on the calendar, one for each role. Another option might be to fill out only one line per person, but then it would become difficult to input an appropriate role.
4. R4 was a very challenging respondent. Although intelligent, R4 appeared thought-disordered, and she was particularly motivated to discuss her personal problems concerning her children. The household roster and the screener questions took a great deal of time. I decided not to follow the scripted questions while going through the calendar, as she had already shown impatience with the earlier questions. I was able to fill out two calendars for her biological children (the daughter had been placed in a different foster care home than her sons). Additionally, I returned to the script to ask about the causes of separation and living with other adults (MOM3, DAD3, OTHAD5), the number of moves, the hypotheticals, and the final debriefing questions concerning the calendar.
5. To my surprise, most respondents were confident in reporting the moves of children, although it was clear that R2 missed one of the moves which was mentioned while filling out the calendar.
IV. Overall Recommendation
In general, I found the scripted portions of the EHC frustrating. The screener questions are too involved, and they should be used only to the extent that they identify eligible children. As is, the interview length will be considerable if for every child, one has to ask questions about both the biological mother and father, and each of the other adults listed in SCR1C. Additionally, the linear flow of the MOM1 to OTHAD5 section interferes with the ability of the calendar to optimize autobiographical recall. In particular, this linear flow discourages respondents to work across the timelines, and to tell about their experiences in a natural, narrative manner. In methodological production setting work conducted at the University of Michigan, interviewers preferred administering a flexible calendar than asking scripted questions, and that the length of interviews was significantly shorter with the calendars. Accordingly, I recommend that the calendar portion of the interview be highlighted as a data collection instrument in its own right. Since there are likely to be a great deal of variation in which aspects of the question objectives are pertinent to individual respondents, interviews will be most efficient by permitting flexible interviewing by using the calendar as the primary guide to collecting information.
The following guidelines deserve to be cognitively tested. Although my last interviews followed these guidelines with no observable major problems, both respondents had rather uncomplicated histories.
1. Use the screener questions only to the extent to which eligible children are identified. For most eligible children then, their eligibility will be decided with SCR1A and SCR1B. If necessary, SCR1C can be used, but present as a flashcard, and allow the respondent to decide with only a single omnibus "yes" (eligible) or "no" response. The screener question can be modified as "was there ever a period of three months of more when (child) lived with any one of the following people?"
2. CKMF should function as well with the recommended changes as originally. Since all of the information comes from the household roster and cycling through the OTHCHLD section, the changes to the screener questions (above) will not affect CKMF.
3. SCR1D and SCR1E should be asked after the calendar has been filled out, using information from the calendar. During calendar work, interviewers are likely to discover whether there were more than one stepmother. Or they can be instructed to probe for more than one of each of the categories of stepmother, etc. Of course, for each different person, a line should be reserved on the calendar.
4. The calendar should be introduced with INTROCAL. INTROCAL ought to be reworded to emphasize that the interest is to gain the periods of time when children were living apart from biological parents for three months or more, or with other adults for three months or more. Flexible interviewing should then be encouraged. Interviewers should be trained regarding the objectives of the calendar. The CAI program can prompt interviewers with suggested question wordings for collecting the information that is presently scripted from MOM1, DAD1, and SCR1C. SCR1C should be presented to respondents as a flashcard, and the interviewer should be clear while interacting with the respondent to include all other adults, whether living with either of the biological parents or not. Interviewers can collect information for each role (as depicted on the flashcard), per person, per line. Additionally, for other adults who had served in more than one role, a line can be reserved for each role that was served, and the interviewer can keep track that an individual served more than one role.
5. After the calendar is filled out, the CAI can be used to enter the information that is found in MOMSEP, MOM3, DADSEP, DAD3, OTHADSEP, OTHAD4, and OTHAD5 (presented as a flashcard!). To record information on "other adults,"for each calendar the CAI application can instruct the interviewer to enter the total number of lines filled out (numerically, following the calendar), how many persons served multiple roles, and for each multiple-roled person, which lines were entered. Then, for each line number, information can be collected concerning the type of role, the number of spells, the start and stop times of each spell, who moved, and the reason for living with the other adult. As some of this will be new information, the interviewer will need to be asking the respondent some of these questions. In fact, the interviewer and respondent can interact in the computer entry of all of this information, which additionally may assist respondents' ability to recall.
Report on Cognitive Interview Results for the
Survey of Program Dynamics
Children's Residential History Module
Jennifer Hess
Center for Survey Methods Research
Statistical Research Division
US Census Bureau
April 9, 1999
I. Background
Members of the American Statistical Association/Survey Research Methods Section Working Group on Technical Aspects of the SIPP and the SPD suggested including questions on children's residential history in the SPD during their meeting on May 29-30, 1997. ChildTrends, Inc. (CTI) prepared a draft children's residential history calendar and associated question list and included it in materials submitted for consideration for the 1999 extended measures of children's well-being module. The draft CTI submitted was intended to be administered by paper and pencil. Staff from the Census Bureau's Population Division (POP) revised the children's residential history module with a computer-assisted administration in mind. Staff from the Center for Survey Methods Research (CSMR) tested two different versions of the children's residential history questions developed by POP in two series of cognitive interviews conducted from February-June 1998. Based on the results of that testing, the SPD Steering Committee decided to delay implementation of the children's residential history module until 2000 to allow additional time for development and testing.
Dr. Robert Belli at the Institute for Social Research, University of Michigan, under contract with CSMR developed a residential history calendar and associated question list to be used in an automated questionnaire. The calendar is a paper instrument to serve as an aid to enhance respondent recall of their children's living situations since birth. POP and CSMR staff revised the question list and calendar to be used in two series of cognitive interviews, one at the University of Michigan and one at CSMR. This report includes results from the latter only. Dr.Belli will submit a separate report summarizing results from his testing. Below is a description of the methodology, a summary of major problems identified, and item-by-item results. Recommendations for revisions to the questionnaire will be based on both sets of cognitive interviews and will be submitted under separate cover.
II. Methodology
Jennifer Hess conducted seven concurrent cognitive think-aloud interviews to test respondent comprehension of the children's residential history questions during February 1999. In addition respondents were asked debriefing questions following the cognitive interview about the usefulness of the residential history calendar as an aid for enhancing respondent recall of their children's living situations. The interviews lasted approximately one hour and respondents received $30 for their participation. Four of the interviews were conducted at CSMR's Response Research Laboratory and were video and audio taped with the respondents' permission. Three of the interviews were conducted at sites more convenient for the respondent and were audio taped. The same interview protocol and debriefing questions were used for interviews conducted at CSMR and the University of Michigan. A copy of the questionnaire/cognitive interview protocol and the residential history calendar tested are included as Attachments A and B.
We successfully recruited respondents whose children (or grandchildren who live with them) have lived in a variety of different living situations throughout their lifetimes (see Attachment C for a complete listing). Three respondents are grandparents who are caring for their grandchildren (respondents 1, 5, and 7 on attachment). One of these grandparents was the foster care provider for four of her grandchildren and is now their adoptive mother. Two respondents were single mothers in their mid-20's (respondents 2 and 3 on attachment). One of these respondents lives with her parents; she has one daughter who lives with her and one that lives with the child's biological father. One respondent was a married father of two children (respondent 4). And the last respondent was woman married for the second time who had children from both marriages (respondent 6). Three of the respondents completed 11th grade, one completed high school, one completed two years of college, and the other two had college degrees.
II. Summary of Major Problems Identified
Below is a brief summary highlighting the most important findings. (The item labels from the questionnaire are shown in parentheses.)
2. There is evidence that the questions asking whether there was ever a period of three months or more when the child did not live with the biological mother (SCR1A) or the biological father (SCR1B) are confusing for respondents if the child either never lived with the biological parent of interest or lived with that parent for less than three months, and may result in respondents providing the wrong answer to these critical items.
3. The question asking whether the child ever shared a household with "other adults" for a period of three months or more did not work well, and introductions added during cognitive testing to clarify the question's intent did not help (SCR1C). The intention of the question is to collect data on times the child lived by himself/herself with the "other adult" and times that the child lived with a biological parent and the "other adult." Several problems surfaced during cognitive interviews: 1) Respondents thought the question was asking about times the child lived by himself/herself with the "other adult" and did not report times when the child and a biological parent lived with the "other adult;" 2) Respondents thought the questions were asking about persons outside their household, that is, persons other than themselves or their current family members, even if they were, for example, the child's adoptive parent, grandparent, or stepparent; 3) Respondents had difficulty answering this question when a person filled more than one role, such as being a nonmarital partner of a biological parent and later becoming a stepparent.
4.The follow-up questions for "other adults" that ask when the child lived with the "other adult" (OTHAD1), whether the "other adult" was responsible for most of the child's basic needs (OTHAD3), the direction of the move (OTHAD4), and the reasons for living with the other adult (OTHAD5) were problematic for persons who filled multiple roles for several reasons. First, there is no way in the current instrument to differentiate between one person who fills multiple roles (e.g. a nonmarital partner who became a stepparent) and two people filling two different roles (e.g. one person who was a nonmarital partner and a second person who was the stepfather) making it impossible to assess the number of different people with whom the child has lived. Persons who fill multiple roles will appear as changes in living situations when, in fact, no change has occurred. Second, some respondents reported the time periods the child lived with the "other adult" in terms of the person rather than the role, some reported in terms of the role, and others used both methods during a single interview. This results in inconsistent data and also leads to respondent confusion in subsequent questions. Third, asking follow-up questions for each role the person filled seems redundant and overly burdensome to respondents.
5. Rules for grouping children who have always lived together on a calendar require that both the child's biological mother and father be in the household before children are potentially eligible to be grouped on the same roster (check item CKMF). These rules may be overly restrictive and result in unnecessary respondent burden. Siblings may have always lived together even though the parents don't reside together in the household. (Grouping children who have lived together is done for efficiency. Questions are asked about the oldest child and data for younger children are plugged based on the child's date of birth.)
6. Following the residential history calendar, we asked a series of questions on moves during the child's lifetime (MOVE1, MOVE2, MOVE3, and MOVE4). These questions are asked by aggregated age groups: prior to elementary school (ages 0-4), elementary school (ages 5-11), middle school (ages 12-14), and high school (ages 15-18). Results indicated respondents had trouble with the reference periods in these questions, which resulted in double counting of moves.
Provided below are the results of cognitive testing for the seven interviews CSMR staff conducted. The question wording tested is provided along with a summary of problems identified. The label name from the questionnaire is shown in parentheses.
I have listed the following children born since January 1, 1974
who are living in this household.
(Do you/Does name) have any other biological children who were born since January 1, 1974 and are living somewhere else?
Problems identified:
Respondents understood that the question was asking
about biological children that
don't live with them. Two respondents, however,
didn't hear the date and reported
children that were born before 1974.
When there were several persons age 15 or over
in the household, this question and
a subsequent one on adopted children living outside
the household seemed verbose
and repetitive to the interviewer and may be better
handled in a topic-based
approach rather than a person-based approach, or
combined into a single item.
How many other biological children (do you/does name) have?
All three respondents who answered this question
had already provided the
relevant information in response to the previous
item. Two of the three respondents
were confused by the question. One respondent thought
the question was asking
how many biological children she had, rather than
how many other biological
children she had living outside the household.
The other respondent thought this
question was asking for new information since she
had already told me how many
biological children she had living outside the
household in the previous item and
thought that the question might be asking about
children given up for adoption or
possibly miscarriages.
ASK IF NECESSARY: What is [his/her] sex?
When was [she/he] born?
Problems identified:
No problems were identified with the items on
the child's name or sex. The date of
birth given for one of the children was July 17,
1971, which is out of scope.
Problems identified:
No respondents had adopted children living outside
the household. Two
respondents thought the question was asking whether
they had any adopted children
(rather than adopted children living outside the
household).
What is [the first] child's name?
ASK IF NECESSARY: What is [his/her] sex?
When was [she/he] born?
Problems identified:
These questions were never asked. See problems
listed under comparable items (2
and 3 above) for biological children living outside
the household.
Problems identified:
This question was somewhat problematic for two
respondents. One of the
respondents was confused by this question because
in item 1 she had already told
me that she has a daughter who lives with her biological
father and this question
sounded redundant to her. In the end, this respondent
answered the question
correctly. Another respondent who is the grandparent
and adoptive mother of her
grandchildren answered this question incorrectly
for two of her grandchildren. The
children were taken away from the biological mother
before they were three months
old and given to the respondent. The respondent
answered "no" to this item because
the children never lived with the mother for a
period of three months.
Problems identified:
This question was problematic for three respondents.
The respondent who was the
adoptive mother of her grandchildren answered this
question incorrectly for three
of her four grandchildren. She answered "no" because
the children had either never
lived with their biological fathers or had lived
with them for a short period of time
(less than 3 months). She had great difficulty
understanding this question and
paraphrased the question as "They want to know
if he ever did live with his
biological father." A second respondent answered
"yes" to this item, when the
answer actually was "no." This respondent is divorced
from the child's father and
the biological mother is not happy with the current
living arrangement (the child is
spending most nights at her father's). She explained
"I probably answered 'yes' out
of defense." A third respondent thought the question
was only asking about past
living situations of the children and not their
present living situation.
1. An adoptive mother
2. An adoptive father
3. A stepmother
4. A stepfather
5. A nonmarital partner of a biological parent
6. A grandparent
7. Some other adult relative such as an aunt or uncle
8. Some other adult non-relative
9. Foster care arrangement
10. An institution, boarding school, group home, or hospital
Introductions tested:
1. None.
2. Now we are interested
in periods of three months or more when (child) may
have
also lived with other adults. Include times when (he/she) lived with
these
other
adults even if (his/her) biological mother or father was with (him/her).
3. We are interested in periods of three months
or more when (child) shared a
household with adults other
than (his/her) biological mother or father, or adults
in additional to (his/her)
biological mother or father. Please include times in
which (child) moved in
with another adult and times when another adult moved in
with (child), even if (he/she)
was still living with (his/her) biological mother or
father.
Problems identified:
This item is intended to measure periods of three
months or more when the child
lived with other adults regardless of whether a
biological parent(s) was present.
After conducting the first two cognitive interviews,
it was apparent that
respondents thought this item was asking about
times that the child went to live with
another adult without a biological parent. Owing
to this, we tested two different
introductions in the remaining cognitive interviews
to attempt to clarify that we are
interested in periods that the child lived with
other adults even if a biological
parent was present. These two introductions are
shown above. Neither introduction
helped matters much as most respondents still interpreted
the question as asking if
the child had lived with these other adults without
a biological parent present. One
respondent thought the introduction was the question
and responded "no" before any
of the subsequent categories was read. Of seven
respondents, only one answered
this item correctly. This respondent's children
had the least complicated living
situation of any interviewed since the children
had always lived with both
biological parents, but the family had moved in
with "other adults" on two
occasions.
Another problem with this series is that respondents
thought the questions were
asking about persons other than themselves, or
persons outside the child's current
living situation. For example, one grandparent
who is the adoptive mother of
grandchildren in her household responded "no" to
the "adoptive mother" and
"adoptive father" items in the list even though
she is the adoptive mother and her
husband is the adoptive father of the grandchildren.
Another grandmother whose
grandson has lived with her since birth also thought
these items were asking about
persons other than herself. She responded "yes"
to "grandparent," but that was
because she was thinking about her nonmarital partner
because he is like the child's
grandfather. She did not think the question was
asking about her. A third respondent
who is remarried reported "no" to "stepfather"
even though her two daughters lived
with her and her second husband. A fourth respondent
reported "no" to grandparent"
even though she and her daughter currently live
with the child's grandparents.
Problems arose when one person filled multiple
roles. In the current instrument,
persons who fill multiple roles appear as separate
lines on the calendar. There is
no way to tell whether one person is filling more
than one role or whether each role
is a separate person. Among the respondents interviewed,
we had several situations
in which one person filled multiple roles: 1) a
couple who filled four roles--
grandparent, foster care arrangement, legal guardian,
and adoptive parents; 2) a
live-in ex-husband of the respondent filled two
roles, grandparent and nonmarital
partner; 3) two stepfathers who were the nonmarital
partners of the biological
mothers prior to marriage; 4) a stepmother who
was a nonmarital partner of the
biological father prior to marriage. Problems were
compounded in follow-up
questions regarding when the child lived with these
people, whether they were
responsible for most of their basic needs, the
direction of the move, and the reason
for the move. These questions are asked by role.
Some respondents answered in
terms of the person, rather than the role. Others
answered in terms of the role, and
some applied both methods during a single interview,
as their understanding of the
intent of the question changed during the interview.
Asking these questions by role
caused misreporting and also lead to additional
respondent burden.
respondents. One respondent had no idea what this term meant. One said she didn't
know but guessed that it meant "girlfriend" or "boyfriend." Another respondent was
unclear how to respond since the child's biological mother and stepfather weren't
married initially. She eventually responded "no" to "unmarried partner of a
biological parent." Another respondent said "When you said >nonmarital partner' I
was thinking >boyfriend,' but then you said "of a biological parent,' which would be
like my spouse." This respondent was unclear how to respond to this item. She and
her second husband weren't married when their first child was born, but they were
both his biological parents. Another respondent reported "yes" to this item, but was
referring to her nonmarital partner. She is the child's grandmother, not the
biological mother.
Problems identified:
No problems were identified.
Problems identified:
Only one person answered this question. No problems
were identified.
Have (name of younger child) and (name of older child) always lived in the same household without a separation of three months or more?
Problems identified:
Only one respondent went through this item. None
of the other respondents lived in
households in which both biological parents were
present as required in the check
item (CKMF). No problems were identified. There
were several other children that
had always lived together, but were not able to
be included on the same calendar
since both biological parents were not in the household.
These included: 1) one
respondent who had three biological daughters who
had always lived together, but
the children's father did not live with them, 2)
one respondent who was raising her
two grandsons who had always lived together, but
neither biological parent lived in
the household currently, and 3) the adoptive mother
of four grandchildren that could
have been on two calendars instead of four since
two of the children had always
lived together and the other two had always lived
together.
Problems identified:
This introduction was added part way through testing.
In the first couple of
interviews, respondents paid no attention to the
residential history calendar and did
not use it to help them remember dates. This introduction
was added to give
respondents a better understanding of the calendar
and to involve them as
participants in filling it out. Due to interviewer
oversight, respondents were not
probed regarding their understanding of the introduction.
Based on interviewer
observation, some of the respondents who were read
the introduction did seem to
take more interest in the calendar than those who
were not read the introduction.
Problems identified
Five of the seven respondents had children who
had been separated from their
biological mothers. Three of the respondents had
some trouble with dates of
separations, but did their best to come up with
dates. Respondents usually reported
only the beginning dates and did not report end
dates to this item. Field
Representatives (FRs) will need to be trained to
probe for end dates.
The one respondent who had a joint custody arrangement
for her two children
reported them as living with both parents when
the children spent parts of each
week with each parent, but then reported one daughter
as living only with the
biological father when she started spending most
of her nights at the biological
father's house and only spending a few nights a
month with the biological mother.
The respondent reported that this shift from a
joint arrangement to one in which the
child spent more time at the father's house happened
gradually over a period of
time, which made it difficult to give a date that
the arrangement changed.
Note that the interviewer made a mistake in one
interview regarding the date of
separation from the biological mother. The respondent
said it occurred when the
child was age one in about June. The interviewer
looked at the calendar for the
year in which the child was age 1 and wrote "6"
for June under that same year. The
"6" should have gone in the following year. FRs
should be trained that children are
age 1 up until they turn 2 the next year and to
be careful not to mark the wrong year.
Problems identified:
One respondent (a grandmother caring for her grandson)
reported that the mother
moved out to "be with her boyfriend and develop
their lives." A few years later the
mother had a car accident resulting in a head injury
and now requires round the
clock care at a group home. She wasn't sure whether
she was supposed to report
both reasons since they happened at different times.
Another respondent couldn't
read the flashcard but said "child abuse" in response
to this question. The
interviewer read the category "child removed by
government agency," and the
respondent initially said "yes," but then was unsure
because she said that Social
Service is run by the State. (Perhaps she equated
"government agency" with the
Federal government.) Another respondent chose "parental
financial hardship"
because the child "had it better at her dad's."
Is this an intended meaning of
this category?
The wording of this question is awkward if there
is only one separation. The
interview reworded the question as follows: "What
caused the separation between
(date) and (date)?"
Based on a respondent's misunderstanding of the
comparable item for separations
from the biological father, this item should specify
that we are talking about a
separation from the biological mother.
Problems identified:
Five of seven respondents answered this questions.
Respondents often gave the
start date for the separation but not the end date.
FRs will need to be trained to
probe for end dates.
One respondent (the grandmother of the child)
had trouble answering this question
about her nonmarital partner's biological son.
This child had never been a member
of this household. The best answer she could give
is the first quarter of 1982. FRs
will need to know how to code such answers.
Problems identified:
One respondent said that the "parents were no
longer compatible." The interviewer
asked her to choose a category from the flashcard
and the respondent chose "other
reasons" rather than "marital conflict between
parents." This suggests that the
flashcard may be too long for respondents to read
easily and quickly. Another
respondent was confused by this question on two
separate occasions. One of her
daughters lived with her when she was younger but,
after a custody battle, went to
live with the child's biological father. The respondent
thought this question was
asking why the child left the mother to go live
with the father. The younger daughter
of this same respondent has never lived with her
biological father and the
respondent was confused by the question because
"there never was a separation
because she never lived with him." A third respondent
had trouble with this
question because she and the children's father
were never married and she wasn't
sure whether she should use the category "marital
conflict between parents" since
the parents weren't ever married.
The wording of this question is awkward if there
is only one separation. The
interviewer reworded question as "What caused the
separation between (date) and
(date)?" Also, the question should specify that
we are asking about separations
from the biological father.
Problems identified:
Many of the problems with this item stem from
respondents' misunderstanding of
the previous "screener" item regarding "other adults"
with whom the child has
shared a household. First, respondents who thought
the screener was asking only
about times that the child went by himself/herself
to live with the "other adult," did
not mention times that the biological parent(s)
and the child lived with the "other
adult." The grandmother who lives with her two
grandsons failed to mention the
two times the grandchildren and their biological
father lived with the grandparents
and only reported the times that the grandchildren
lived with the grandparent when
the biological father was not present. Two other
respondents failed to mention
times when they and their children lived with the
respondents' parents (child's
grandparents). One respondent failed to mention
the times she and her two
daughters lived with three different nonmarital
partners. One respondent did not
report the time the child's uncle lived in the
household for a year and a half.
Second, respondents who thought the question was
asking about persons other than
themselves or persons outside their current living
situation that the child lived with
were excluded from this item altogether. There
were several examples of this from
the cognitive interviews. Two separate grandparents
didn't include themselves in
the screener item. One of the grandparents was
the children's adoptive mother and
her husband was the adoptive father. One respondent
reported "no" to "stepfather"
even though her two daughters lived with her and
her second husband. One
respondent reported "no" to grandparents even though
she and her daughter live
with the child's grandparents.
a child's life. In some cases the respondent reported the time periods the child lived
with the "other adult" in terms of the person and did not differentiate between the
roles; in other cases, the respondent reported the time periods in terms of the
role; and respondents sometimes applied both methods during the same interview
as their understanding of what they thought we wanted changed. This results in
inconsistent information.
We may need to clarify that the questions are
asking about the period of time from
birth to present (or to age 18 for persons currently
over 18). One respondent failed
to mention that she and her baby and the baby's
father lived with the child's
grandparent until the child was age one because
it happened a long time ago and
she thought the questions were asking only about
more recent events.
One respondent could give the year that the son
of her nonmarital partner started
living with a stepfather, but not the month. (The
son has never been a member of
this household.) FRs will need to be instructed
how to code such information on the
calendar.
Additionally, respondents tended to give start
dates but not end dates. FRs will
need to be trained to probe for end dates.
Problems identified:
This question worked fairly well. Respondents
generally agreed on what the
term "basic needs" meant and included things
like food, clothing, shelter,
medical care, education. However, respondents
had different interpretations
of the concept of being "responsible for most
of the basic needs." Some
respondents thought of this in terms of financial
responsibility. Other
respondents indicated that it took more than
money to raise a child and thought
about everything that it takes to raise a child.
If the intention is to find out who
was financially responsible for the child, then
the word "financially" should be
inserted into the question.
One respondent had trouble answering this question
with regard to the time
that her daughter lived with a stepmother. The
respondent has a joint custody
arrangement for her daughter. For most of the
child's life, responsibility for
providing the basic needs of the child was shared
between the two families.
Later the daughter spent more time at the father's
house and he and his wife
provided more for the daughter. The respondent
wanted to provide two
answers. One from the time of the separation
until the daughter started living
with the biological father and his wife (when
custody was shared) and a
second for the time that the child lived with
primarily with the biological
father and his wife.
Problems identified:
This question did not work well. In a couple
of instances, the child was a
newborn and came home from the hospital to the
residence. This question was
awkward for those circumstances mostly likely
because respondents don't
think of children as "moving" when they come
home from being born.
This question also did not necessarily prompt
respondents to think about
changes in the initial living arrangement if
something more significant
occurred later like moving to a new house. For
example, one respondent
reported that the child and the grandparent moved
together to a new place.
This happened 9 months after the child was born.
When the child came home
from the hospital, she went with the mother to
live in the grandparents house
so the correct answer should have been "child
moved." Another respondent
reported that they moved to a new house, when
in fact the nonmarital partner
had moved in with her initially and later they
moved to a new house.
This question also did not work well in situations
in which the person's role
changed. For example, one respondent lived with
her husband before they
were married. His role changed from "nonmarital
partner" to "stepfather."
When this question was asked about the stepfather,
the respondent didn't know
how to answer because they were all living together
and no one moved; just
the stepfather's role had changed from nonmarital
partner to stepfather.
Problems identified:
This question is similar to the two other questions
asking for reasons the child
was separated from the biological parents, but
no flashcard was used. Both
questions should be consistent by either using
or not using flashcards. A few
respondents thought they were supposed to use
the flashcard from the previous
question on reasons for separation from a biological
parent to answer this
question.
This question is awkward when the reason a child
and the "other adult" started
living together was because the child was born
and came home from the
hospital. At a minimum, the category "child born"
should be added to the
response options.
This question was confusing for some respondents,
particularly when a
biological parent and the child lived with the
"other adult." This prompted
responses of "because that's where I was living"
or "that's where his mom was
living." In these instances, the question really
should be why did the parent
start living with the "other adult" since the
reason the child is living there is
because that's where the parent is living.
This question also caused problems when persons
filled multiple roles and
answered the previous item OTHAD1 (time periods
child lived with the
"other adult") based on the person rather than
the role. For example, one
respondent answered OTHAD1 for the entire time
her daughter lived with the
person who eventually became her stepmother.
This included time when the
stepmother was the nonmarital partner of the
father. The month/year fill for
this item is based on data from OTHAD1. Thus,
the date and the role did not
match, since the date corresponded to the month
and year the daughter started
living with the father and his nonmarital partner
and the role corresponded to
"stepmother." The respondent pointed out the
inconsistency. In a standard
interview, such errors would require the FRs
to back up and change answers
to several previous items.
The following questions on moves are not included as part of the residential history questions. They are a separate series of questions that were tested after all the questions pertaining to the residential history were completed for all eligible children. The objective of the questions is to provide the number of moves a child made with or without other family members during the child's lifetime. The questions are asked for aggregated ages that correspond to elementary, junior high, and high school, as well as prior to entry in school. (Wording shown in brackets is for children younger than the maximum age for the question.)
1. Tested wording (MOVE1)
Problems identified:
The question reads "with or without you." Depending
on who the respondent
is, the "you" refers to different people. If
the respondent is not a biological
parent, should the question be filled "with or
without a biological parent?"
Several respondents did not hear the reference
period in this question, which
resulted in double counting moves. Additionally,
the phrase "Before he/she
began school," may be misleading respondents.
Two respondents thought it
was asking how many times the child moved since
starting school. Another
missed the reference period altogether and reported
the total number of moves
the child made during his lifetime.
Respondents may not know this information for
children that have never been
part of their household. One respondent had trouble
answering this question
for the biological son of her nonmarital partner.
She has never lived with this
child so wasn't really sure when he and his biological
mother moved.
Problems identified:
At least one respondent double counted a move
both at this question and the previous one. One
respondent had difficulty answering this question
for her 19 year old daughter because she had to think
about all the moves she made during those years
and all the moves her ex-husband made because they
shared custody of the child. The school reference
was helpful for one respondent because she knew that
her grandson changed schools every year until
5th grade. The other respondents relied more on the age
of the child rather than the reference to "elementary
school."
Problems identified:
One respondent reported a move that was outside
the reference period (when the child was 16).
Problems identified:
The same respondent who reported a move that
was outside the reference period in the previous item,
neglected to report that move at this item. Another
respondent double counted a move both in the
previous item and here.
Moves
Following the questions on moves, we asked a series of hypothetical vignettes asking about different situations and whether respondents thought the situation should be reported as a move based on their understanding of our definition of a move ("by move I mean to move to a new address, either with or without you for a period of three months or more.") The results of the vignettes following.
Three of the vignettes dealt with situations in which children shuttled between divorced parents. In one vignette, the child lived with the mother during the school year and with the father for three months in the summer. In a second, the child lived in the same arrangement but only stayed with the father for two months in the summer. In the third, the child spent every other weekend with his father and the rest of the time with his mother. The majority of respondents would not report such situations as moves. They characterized them as visits and just temporary in nature, rather than as permanent moves. Respondents also characterized the vignette in which the child lived with the grandmother during the week to attend school and with her parents on the weekends as a visit rather than a move.
Five of seven respondents said they would count the vignette in which the child was at boarding school during the school year and at home during the summer as a move. Two said they would not. And all but one of the respondents thought that a move from one apartment to another in the same building should be counted as a move since it involves different addresses.
Opinions about the residential history calendar
Following the cognitive interviews, we asked respondents several questions regarding their impression of the residential history calendar. Respondents' overall impression of the calendar varied. Some thought the calendar would be a useful tool for respondents even if they themselves did not use it to help them answer questions. Other respondents thought the calendar looked busy, confusing, and unorganized. For example, one respondent didn't understand why "child 1" and "age" and "grade" appeared under the columns marked 1994, 1995, and1996 at the top of the calendar. This respondent also wondered why there was no place to put the child's birth date on the form. (The interviewer put the birth date on top of the first row in order to fill in the children's correct ages in the boxes. The interviewer was unable to put the child's age and grade in the box due to space limitations.) A few respondents also thought that there were too many numbers at the bottom of the calendar. They are referring to the small numbers within each box that correspond to the number of the "other adult" on the far right-hand side. The interviewer did not find these numbers particularly helpful and managed to enter data in the wrong line anyway. One respondent indicated that her eye is drawn to the left-hand side of the page (where we normally start reading) and not to the right -hand side of the page where the children's names and the "living with/apart" information is listed. She suggested putting that information on the left-hand side so that she could read across from left to right. Two respondents thought that the information would be more easily seen and understood if it were more like a bar graph. One suggested using high lighters to make it more visually dramatic. Older respondents also had great difficulty seeing the form due to its small print.
There were mixed results regarding whether the form helped respondents remember dates. Some respondents knew the dates of changes in their children's living arrangements and did not find the calendar useful. Other respondents thought that it was helpful to have it in front of them and that it might be useful for multiple children within a household. None of the respondents knew what the numbers that the interviewer had written in the boxes represented. (The numbers correspond to the month that a change in the living situation occurred.)
DRAFT–INTERVIEWING PROTOCOL
SPD 2000 CHILD RESIDENTIAL HISTORY QUESTION LIST
[Read bracketed introduction once per household.]
[Cycle <OTHCHLD1> through <CKSCR1A> for each person age 15 or over in household.]
<OTHCHLD1>
[In this section, we would like to ask some questions about the past and present living arrangements of the children of this household.
I have listed the following children born since January 1, 1974 who are living in this household.]
Name Sex Birthdate Relationship to Resp
{LIST ALL CHILDREN BORN SINCE JANUARY 1, 1974 WHO ARE LIVING IN THIS HOUSEHOLD.}
<1> Yes
<2> No (go to <OTHCHLD4>)
<D> Don’t know (go to <OTHCHLD4>)
<R> Refused (go to <OTHCHLD4>)
Probe:
What is this question asking in your own words?
What does the phrase “biological children” mean to you?
________________________________________________________________________
<OTHCHLD2> How many other biological
children (do you/does name) have?
ENTER NUMBER OF CHILDREN
________________________________________________________________________
<OTHCHLD3> What is [the first] child’s name?
@1 __________________________________
ASK IF NECESSARY: What is [his/her] sex?
<1> Male
<2> Female
@2 ==>
When was [she/he] born?
@3 ___Month ____Day ____Year
________________________________________________________________________
[Ask if person is 18 or over. Otherwise go to CKSCR1A.]
<OTHCHLD4> (Do you/Does
name) have any adopted children who were born since January 1, 1974 and
are
living somewhere else?
<1> Yes
<2> No (go to CKSCR1A)
________________________________________________________________________
<OTHCHLD5> How many
adopted children living elsewhere (do you/does name) have?
ENTER NUMBER OF CHILDREN.
________________________________________________________________________
[Cycle through <OTHCHLD6@1-@3> until information for all additional
adopted children has been collected.]
<OTHCHLD6> What is [the first] child’s name?
@1 ____________________________________
ASK IF NECESSARY: What is [his/her] sex?
<1> Male
<2> Female
@2==>
When was [s/he] born?
@3____Month ___ Day ____ Year
________________________________________________________________________
CKSCR1A. Go to <OTHCHLD1> and ask for next person in household age
15 or over.
If no additional person age 15 or over in household, go to <SCR1A>.
________________________________________________________________________
[Ask <SCR1A> through <SCR1C> for each eligible child beginning with
the oldest eligible child.]
[Fill bracket if child is 18 or over. Otherwise leave blank.]
<SCR1A> [Up
until he/she was 18,] Was there ever a period of three months or more
when (child) did not live
with (his/her) biological mother?
<1> Yes
<2> No
<D>
<R>
Probe:
What is this question asking in your own words?
________________________________________________________________________
[Fill bracket if child is 18 or over. Otherwise leave blank.]
<SCR1B> [Up
until he/she was 18,] Was there ever a period of three months or more
when (child) did not live
with (his/her) biological father?
<1> Yes
<2> No
<D>
<R>
Probe:
What is this question asking in your own words?
________________________________________________________________________
[Fill bracket if child is 18 or over. Otherwise leave blank.]
<SCR1C> Intro 1:
Now we are interested in periods of three months or more when (child)
may have also
lived with other adults. Include times when (he/she) lived with these
other adults even if
(his/her) biological mother or father was with (him/her).
Intro 2: We
are interested in periods of three months or more when (child) shared
a household
with adults other than (his/her) biological mother or father, or adults
in additional to
(his/her) biological mother or father. Please include times in which
(child) moved in with
another adult and times when another adult moved in with (child), even
if (he/she) was
still living with (his/her) biological mother or father.
[Up until he/she was 18,] Was there ever a period of three months or more when (child) lived with:
READ CATEGORIES
an adoptive mother
<1> Yes <2> No
an adoptive father
<1> Yes <2> No
a stepmother
<1> Yes <2> No
a stepfather
<1> Yes <2> No
a nonmarital partner of a
biological parent
<1> Yes <2> No
a grandparent
<1> Yes <2> No
some other adult relative
such as an aunt or uncle
<1> Yes <2> No
some other adult non-relative <1> Yes
<2> No
foster care arrangement
<1> Yes <2> No
an institution, boarding school,
group home, or hospital
<1> Yes <2> No
Probe: (Re-read question)
What is this question asking in your own words?
What do the terms “stepmother” and “stepfather” mean to you?
What does the term “a nonmarital partner other than a biological
parent” mean to you, if anything?
What does the term “some other adult non-relative” mean to you?
Could you give me an example of someone who would be in that
category?
Do you think this question is asking about times that the child went to
live with the “X” or times when “X” came to live with the child or
both?
(For each “yes” ask) Did child go to live with “X” or did “X” come to
live with the child?
________________________________________________________________________
CK1.
If stepfather, stepmother, non-marital partner or foster care marked “yes”,
go to <SCR1D> for each “yes”
response.
If stepfather, stepmother, non-marital partner and foster care equal “no,”
“D,” or “R” for all responses, go to
CK2.
________________________________________________________________________
[Cycle <SCR1D> through <SCR1E> for each “yes” response in <SCR1C@3-@5>
and <SCR1C@9>.]
[Fill bracket if child is 18 or over. Otherwise leave blank.]
[Fill “live with” if asking about <SCR1C@3-@5>. Fill “live in”
if asking about <SCR1C@9>.]
<SCR1D> [Up until he/she was 18,] Did (child)
(live with/live in) more than one stepmother/stepfather/nonmarital