U.S. Census Bureau Homepage

Survey of Program Dynamics

Report on Cognitive Interview Results

for the

1999 Survey of Program Dynamics

Jennifer C. Hess

Jennifer M. Rothgeb

Elizabeth M. Nichols

 

Center for Survey Methods Research

Statistical Research Division

Bureau of the Census

June 17, 1998

Table of Contents

I. Background

II. Methodology

III. Cognitive Test Results: Child Well-being Module


A. Positive Interaction
B. Cognitive Stimulation
C. Enrichment Activities
D. Positive Behaviors/Social Competence
E. Grades/Achievement
F. Household Resources
G. Conflict Between Parents Who Co-reside
H. Conflict Between Parents Who Do Not Co-reside
I. Family Conflict
J. Attitude Toward Welfare/Work
K. Knowledge About Welfare Regulations

IV. Cognitive Test Results: Expanded Content for the SPD Core Interview

A. Child Care Subsidies
B. Child Care Problems
C. Quality of Child Care
D. Employee Fringe Benefits
E. Health Care Utilization While Uninsured
F. Consumer Expenditures

1.Food Expenditures
2. Transportation Expenditures
3. School Expenditures
4. Health Insurance Expenditures

G. Months Receiving Rental Assistance

V. Cognitive Test Results: Children's Residential History

A. General comments
B. Results of Round 1 Testing
C. Results of Round 2 Testing

Attachment A Development Plan (time line) for SPD99
Attachment B Objectives for the Child and Family Well-Being Questions
Attachment C Content Areas Not Cognitively Tested by CSMR

Report on Cognitive Interview Results - SPD99

I. BACKGROUND

Early in the development of the Survey of Program Dynamics (SPD), the SPD Steering Committee envisioned the survey as composed of two parts: a 60 minute "core"interview administered annually starting in 1998, and a 30 minute child well-being module to be added in 1999. In summer 1997, scarcity of resources (staff and monetary) and the need to focus efforts on ensuring the success of the core SPD in order to meet legislative goals, prevented moving forward with the original plans for a child well-being module. The SPD Steering Committee reexamined resources and determined that a 5-10 minute child focused module was a realistic alternative.

In Fall 1997, the Bureau of the Census solicited input from the SPD Interagency Committee for a 5-10 minute child focused module. A development plan (Attachment A) was circulated, soliciting content and identifying dates for subject matter to be proposed, tested, and evaluated. Solicitation for content was eventually expanded to cover not just the child well being module, but the core SPD instrument as well. In addition to government agencies, input was solicited from data users including those from academia and the private sector. Proposed content was received from the Family and Child Well Being Network of the National Institute of Child Health and Human Development (NICHD), Child Trends, Inc. (CTI), the Department of Health and Human Services (HHS), the Department of Agriculture, the Office of Management and Budget (OMB), Manpower Demonstration Research Project (MDRC), the Department of Health Care Policy at Harvard University, and the Population Division (POP), and the Housing and Household Economic Statistics Division (HHES) of the Bureau of the Census. The SPD Steering Committee decided that all proposed topics be considered for the module and, in accordance with the Census Bureau's pretesting policy, required testing be conducted for appropriate question series.

Proposed content includes the following:

Child Well Being

Positive interaction
Cognitive stimulation
Enrichment activities
Positive behaviors/social competence
Grades/achievement
Household resources Family conflict
Conflict between biological parents who co-reside
Conflict between biological parents who do not co-reside
Attitudes towards welfare/work
Knowledge about welfare regulations
Children's residential history
Family routines
School engagement
Behavior problem index
Aggravation in parenting
Children's routines
Substance abuse

In addition to the child well-being module, proposed expanded content for the SPD core instrument includes the following:

SPD Core expansion

Knowledge of child care subsidies
Child care problems
Quality of child care
Employee fringe benefits
Health care utilization while uninsured
Consumer expenditures (food, transportation, school, health insurance)
Months receiving rental assistance
Coping mechanisms when losing welfare benefits

We requested that persons proposing content for SPD99 provide specific question wording and the universe for whom the questions applied, along with the objective of the question or question series, background on results of prior use (e.g. item nonresponse), and, if available, the results of prior testing. This information was needed to determine whether results of prior testing or prior use demonstrate that proposed question series "work "and don't require further testing. Additionally, information on the objectives of the questions aid in development of appropriate probes for the test protocol to indicate whether respondents understand the question as intended and ensure that the objective of the question is being met. Objectives provided for some of the proposed content are contained in Attachment B.

Based on review of limited documentation and discussions with subject matter experts and survey methodologists, CSMR determined that the following proposed SPD99 content did not require testing because there was enough evidence from prior use and/or testing that the items "worked."

Proposed content areas NOT cognitively tested by CSMR

(Questions for these series are contained in Attachment C.)

Aggravation in parenting
School engagement
Family routines
Children's routines
Mental Health Indicator (Behavior problems index)
Food Bank
Coping mechanisms when losing welfare benefits
Substance abuse

The SPD Interagency Committee agreed that, once testing was completed and evaluated, content areas would be prioritized and selected for inclusion in the SPD99 instrument based on discussions and decisions reached within the SPD Interagency Group. This report documents the results of the cognitive laboratory testing conducted with the proposed SPD99 content.

II. METHODOLOGY

We conducted thirty-four concurrent cognitive think-aloud laboratory interviews to test respondent comprehension, task difficulty, and item sensitivity of proposed questions. Three researchers from the Center for Survey Methods Research (CSMR) conducted the cognitive interviews and summarized results. We tried to keep cognitive interviews to no more than one hour to avoid respondent fatigue. Respondents received $30 for their participation. Interviews were conducted in CSMR's Response Research Laboratory, as well as at off-site locations more convenient for respondents. All interviews were audio taped (with written consent from respondents). Due to the magnitude of material proposed for testing and late delivery of required information for some proposed topics (e.g. question wording, objectives, prior use, prior testing), we conducted two rounds of interviews, referred to as Rounds 1 and 2 below.

Due to the quantity of content tested, we minimized the use of direct probes and maximized use of general probes. General probes used included: "How did you arrive at your answer? " "Tell me more about that. " "In your own words, what is this question asking? " During most interviews there was only time to administer the child-focused questions for one child within the household. Efforts were made to balance the ages of the children for whom information was collected.

In Round 1, we included twelve content areas (enrichment, cognitive stimulation, positive interaction, family conflict, parental conflict (co-reside), parental conflict (don't co-reside), household resources, quality of child care, attitudes towards welfare/work, knowledge about welfare regulations, consumer expenditures, and children's residential history). Respondents were recruited primarily from job training programs, GED programs, and other social service programs. Typically, the CSMR recruiter contacted a program director, who then identified potential respondents and scheduled interviews. Round 1 respondents were primarily low-income single mothers, many of whom were current or former welfare program recipients. (None of the respondents was male.) Eighteen Round 1 interviews were conducted March 9 - March 13 with adults with children under 18 living in their households. Children in Round 1 were in the following age groups: 3 children 0-2 years old; 5 children 3-5 years old, 5 children 6-11 years old; and 5 children 12-17 years old. Most of the Round 1 interviews were conducted off-site.

In Round 2, we included seven new content areas (positive behaviors/social competence, grades/achievement, knowledge of child care subsidies, child care problems, employee fringe benefits, utilization of health care while uninsured, rental assistance) as well as revised versions (or subsets) of eight content areas previously included in Round 1 (family conflict, parental conflict (don't co-reside), positive interaction, cognitive stimulation, enrichment activities, attitudes toward welfare/work, food expenditures, children's residential history.) Respondents were recruited through newspaper ads and from CSMR's respondent pool. Sixteen Round 2 interviews were conducted between April 7 and May 5 with adults with children under 18 in their household. Children in Round 2 were in the following age groups: 1 child 0-2 years old; 3 children 3-5 years old, 5 children 6-11 years old; and 7 children 12-17 years old. Most of the respondents were lower-middle income and married. In Round 2, five respondents were males. All of the Round 2 interviews were conducted on-site in CSMR's laboratory. Population Division (POP) staff observed (through a one-way mirror in the observation room) some of these interviews.

III. COGNITIVE TESTING RESULTS - CHILD WELL-BEING MODULE

Provided below are the results of cognitive testing, grouped by content area. In general, for each content area, the tested question wording is provided followed by a description of problems identified through testing, and recommendations for revised question wording or response options. The majority of the time, problems identified are immediately below the tested wording for each specific question, followed by the recommended revisions, as applicable. In some cases, problems identified pertain to a whole series of questions rather than a specific question. In such instances, the question series is provided and "problems (series)"are located at the end of the series. If content was tested in Round 1 and Round 2, then the tested wording and problems identified are labeled specific to the round of testing to which they apply. If there is no recommendation provided after the "problems"section, then it should be assumed that no question wording revision is recommended. We provide the universe for each content area at the beginning of each section. If the universe changes, we note that in the specific item. If a flashcard is intended to be used we also note that at the beginning of the section.

In some series there are not any recommended revisions from Round 2 testing. This is especially true for the enrichment questions and the cognitive stimulation series. Midway through testing the sponsoring division (POP) decided that, in order to retain time series (back to SIPP 92/93 Child well-being module), revisions to these series in 1999 are unacceptable. For these sections, we document whatever problems we detected, but do not provide recommended revisions. This is also true for the residential history section. In late May the Census Bureau decided that the residential history module will not be ready in time for the SPD99, and will be administered instead in the SPD 2000. This allows more time for thorough development. Census and external experts are currently negotiating objectives and design issues for this series. Therefore, we defer recommendations for revisions to that group. We do identify various issues that we think should be considered during development of the module.

A. Positive Interaction

(Universe : 0-11; Flashcard)

Round 1:
Tested wording:

How often do you praise (child) by saying something like "Good for you! "or "What a nice thing you did! "or "Good going! "? (Same response options for all items during Round 1.)

<1> Never
<2> About once a week or less
<3> A few times a week
<4> One or two times a day
<5> Many times each day

Problems:

Nearly all respondents selected the response option "many times each day. " We expressed concern to CTI about the utility of this item if there is no variance in response. We recommended deleting this item (and series). CTI recommended using NLSY HOME wording and retesting.

Recommendation:

Revise question wording: How often do you praise (child) for doing something worthwhile?

Round 2:
Tested wording:

How often do you praise (child) for doing something worthwhile? (Same response options used for all items during Round 2.)

<1> Never
<2> One to two times a month
<3> About once a week
<4> A few times a week
<5> One or two times a day
<6> Many times a day

Problems:

Even though the phrase "doing something worthwhile "has a broad definition, the wording does seem to be an improvement over Round 1 wording since it does focus the intent a little more. A few respondents did mention that the child had to do something "right ", "good ", or "well "for praise, not just praise for the sake of praising. One respondent thought that anything the child did was worthwhile. One respondent said that normal things are worthwhile. One respondent was more restrictive in what was worthwhile, giving examples of chores and educational accomplishments. It seems parents of young children praise them for good behavior, doing things that a parent asks them to do, and for encouragement.

In Round 2, only 5 of the 10 respondents chose "many times a day ". The rest chose response options <4> and <5>. It is unknown whether the question revisions, response revisions, or differences in respondents led to result differences between Round 1 and Round 2.

Recommendation:

Retain Round 2 question wording and response options.

Round 1
Tested wording:

How often do you and he/she talk or play with each other, focusing attention on each other for five minutes or more, just for fun?

Problems:

This question did NOT work. Specifically, respondents did not include only one-on-one time, nor did they use the five minute time frame. Respondents seemed to think the question was asking about "activities and interactions, in general "between the child and parent and most gave high frequency reports. We recommended deleting this item (and series). Child Trends recommended rewording the question.

Recommendation:

How often do you and (child) talk or play with each other, one on one, for five minutes or more, just for fun? Use revised response options, as identified in first item in the series.

Round 2
Tested wording:

How often do you and (child) talk or play with each other, one-on-one, for five minutes or more, just for fun?

<1> Never
<2> One to two times a month
<3> About once a week
<4> A few times a week
<5> One or two times a day
<6> Many times a day

Problems:

This is a slight improvement over the Round 1 wording. The addition of the phrase "one-on-one "helped define that we are asking just about time between the parent and child. Evidence is provided in the paraphrases of "How much time do you spend, just spend with the child, interacting directly with the child, not with everyone else, but just you and the child. " "How much time do we have alone to play or talk with each other. Just the two of us. "

As with Round 1, respondents did not mention five or more minutes in their responses when asked why they picked a particular category. And when probed, many said they had not considered the timeframe. We think that this timeframe should be retained because it may provide a subtle clue that shorter durations are important. Many of the respondents might have used five minutes or more as a benchmark, even though they didn't articulate that is what they did. A number of respondents gave off the cuff answers of "everyday, "which was not part of the response set. Field Representatives (FRs) need to be trained to probe further if this response occurs. Only two respondents chose "a few times a week. " The other respondents chose response options 5 or 6.

Recommendation:

Retain Round 2 question wording and response options.

Round 1
Tested wording:

How often do you and he/she laugh together?

Problems:

Nearly all respondents selected "many times each day. " This item was not tested in Round 2.

Recommendation:

While the question was well understood, we recommended deleting the item (and series) since it seemed that there would be no variance to responses received and we question the utility of the data.

Round 1
Tested wording:

How often do you do something special with him/her that he/she enjoys?

Problems:

This item did NOT work. There was wide variation in the interpretation of the question. Many respondents included everyday, routine-type activities such as "holding and rocking the child, " "remembering little things or making a dessert, or washing her bellbottoms " "taking him to the store to get a treat " "playing a game. " Other respondents thought the question was asking about less frequent, outside the home activities such as "going to get a haircut, just him and me, " "going to Jeepers, " "going to Chuckie Cheese, " "going to the movies, etc. " When probed about the phrase "that he/she enjoys "some respondents mentioned that not everything that is special is something the child enjoys. When probed about the phrase "special ", respondents seem to think that it meant something out of the ordinary, not routine, something the child likes to do/enjoys, something that means a lot to the child, that would be fun for the child.

We also think the response scale is inappropriate and possibly misleading. The frequencies in the response options seemed to imply that "something special "is an everyday event, given that the low frequency was "about once a week or less, "and the high frequency was "many times a day. " CTI recommended rewording the question.

Recommendation:

Delete this item (and series). CTI recommended revising the question. A revised question was developed. Revised response options were developed that implied expectations for lower frequency reports. Revised question and response options are displayed below.

How often do you do something special with (child)?

<1> Never
<2> One to two times a month
<3> About once a week
<4> A few times a week

Round 2
Tested wording:

How often do you do something special with (child)?

<1> Never
<2> One to two times a month
<3> About once a week
<4> A few times a week

Problems:

This is an improvement over Round 1 wording. Over half of the respondents said that something special was not an everyday event. "Something special "was out of the normal routine. Only one respondent said that an everyday event could be special. "Something special "did have different interpretations however. One respondent said that buying treats was something special; other respondents associated doing something special with weekend times or payday. Two respondents didn't think playing in the backyard or reading a book was something special. Those respondents thought that something special required effort from the parent or something planned ahead of time.

Two respondents were probed about previous wording, "... that she enjoys. "These respondents both thought that the former wording would require more thought, since children don't enjoy everything. One respondent said the former wording left it more open to interpretation since "special "for children might be candy, and that is not necessarily what this parent would consider special.

There were much more varied responses to this question in Round 2, evenly spread among response options 2, 3 and 4. It is unknown whether the question wording revision, response option revisions or differences in characteristics of respondents led to results so different than in Round 1.

Recommendation:

Retain Round 2 question and response categories.

Round 1:
Tested wording

How often do you play games with him/her?

(Universe: Ages <2 years old)

Tested wording

How often do you play sports, hobbies, or games with him/her?

(Universe: Ages 2-11)

Problems:

These two items are redundant with items in the cognitive stimulation section. They were not tested in Round 2.

Recommendation:

We recommend deleting them (and the series.) CTI agreed with recommendation to delete these two items.

Series Comment:

Given the problems with this series, we think it should be a low priority for inclusion in SPD99.

B. Cognitive Stimulation

(Flashcard)

Round 1
Tested wording:

Now I'd like to talk to you about (child's) activities with family members. In a typical WEEK, how often do you (or any other family member):

(Read/Show pictures) books to (child)?

(Universe: read, ages 1-11; show pictures, ages <1)

<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Every day

Problems:

The first question asks about family members reading books to their child. Some respondents reported the amount of time the child read to the parent. Additionally, nearly all respondents chose the "everyday "response option. We question whether there will be any variance in responses obtained. We requested additional information from CTI about the question. CTI responded that the question is asking only about reading to the child, not by the child. They recommend rewording the question so the wording will be identical to the SIPP Child Well Being Module Wording used in the SIPP 92 and 93 panels.

Regarding the distribution of responses, CTI informed us that the experience with the National Evaluation of Welfare-to-Work Strategies Child Outcomes Study, which obtained similar information, showed that there was variation in the survey responses obtained.

Recommendation:

How many times in the past week did you or any family member read stories to (child)?
_______number

Round 2, Version 1:
Tested wording:

The next few questions are about activities you or other family members may do with (child).

How many times in the past week did you or any family member read stories to (child)?

(Universe: Ages 0-11)
_________ number

Problems:

Of the four respondents asked this question, two of them provided qualified answers using the word "about "and one respondent provided a range.

Round 2, Version 2:
Tested wording:

How often in the past week have you or any family member read stories to (child's name) - never, once this week, several times this week, everyday or almost everyday, or more than once a day?

( Universe: Ages 0-11)

<1> Never
<2> Once this week
<3> Several times this week
<4> Everyday or almost everyday
<5> More than once a day

Problems:

When asked about the reference period of the "past week, "two respondents reported that they thought about the past week and that their answers would have differed if they were asked about a typical week. One of these families had a guest and one was on vacation, thus both of the families read less than usual. The answer categories seemed reasonable to respondents. No one picked "more than once a day, "although one respondent did say, "at least once a day. " Also, two of the respondents did include times when the child read back to them.

Recommendation:

Use Round 2, version 2 wording, but change the reference period to "typical week."

Round 1
Tested wording:

How often do you (tell stories/sing songs) to child?

(Universe tell, ages 1-11; sing, ages <1)

<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Every day

Problems:

This item tends to provide information redundant with that obtained from the previous item. Many respondents were including reading books to their child as telling stories. We requested additional information from CTI about the intent of the question and whether or not telling stories provides a different cognitive stimulus than reading books. (We thought perhaps the question could be deleted.) CTI informed us that the purpose of this question is to ask parents about a cognitively stimulating activity that they do with their children that does not require material resources (i.e., books). CTI requested that the question not be deleted. They provided revised question wording to be tested.

Recommendation:

In a typical week, how often do you or any family member make up stories or fairy tales or tell stories about family members or about when you were growing up, to (child)?
_________ number

Round 2
Tested wording:

In a typical week, how often do you or any family member make up stories or fairy tales or tell stories about family members or about when you were growing up, to (child)?

(Universe: Ages 0-11)
_______number

Problems:

It was clear to most respondents that this question did not include opening up a book and reading stories. Also, there is inconsistency between the reference period and response format for this question. We ask "how often ", but the answer format is for "number of times ". The question and response format need to be consistent. The question needs to be revised to be less wordy.

Recommendation:

In a typical week, how often do you or any family member make up stories, fairy tales, or tell stories about when you were growing up to (child)?

<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Everyday

Round 1
Tested wording:

How often do you help (child) to do arts and crafts?

(Universe: Ages 3-11)

<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Every day

Problems:

Many respondents included helping child with homework in response to this question. In some cases, the homework involved arts and craft activities such as drawing, coloring, cutting and pasting, but in other cases it did not. We requested additional information from CTI regarding the intent of the question. CTI responded that the item can include arts and crafts homework, but should not include other types of homework that does not involve arts and crafts.

Recommendation:

Reword the question to read: How often do you help (child) do arts or crafts?

Item was not retested in Round 2.

Round 1
Tested wording:

How often do you involve (child) in household chores, like cooking, cleaning, setting the table, or caring for pets?

(Universe: Ages 3-11)

<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Every day

Problems:

This question worked fine. Common example given by respondents was "cleaning room. " Item was not retested in Round 2.

Round 1
Tested wording:

How often do you play games or do puzzles with (child)?

(Universe: Ages 3-11)

<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Every day

Problems:

Many respondents thought only in terms of board games. Some respondents also considered tickling, wrestling with child, and chasing child through the house to be "games. " We requested additional information from CTI regarding the objective of the of question. CTI responded that the objective of the item is to learn whether parents are involving their children in an activity that is cognitively stimulating. They informed us that activities such as tickling, wrestling, or chasing through the house should not be counted as a "game "in response to this item. CTI provided suggested question rewording. Item was not retested in Round 2.

Recommendation:

How often do you play card or board games or do puzzles with (child)?

Round 1
Tested wording:

How often do you talk about nature or do science with (child)?

<1> Never
<2> Once or twice a week
<3> Three to six times a week
<4> Every day

Problems:

The overwhelming majority of respondents had a very broad definition of "talk about nature. " Most considered anything such as walking outside and having the child ask about leaves, snow, trees, etc. as "talking about nature. " With this broad of a definition, we questioned the utility of the item. We requested additional information from CTI regarding the types of activities that are to be counted in this item. CTI responded that these examples are valid and the question is purposely broad. Item was not retested in Round 2.

Round 1
Tested wording:

In the last year, how often did you or someone in your family visit a library with (child)?

(Universe: Ages 3-11)

<1> Never
<2> A few times during the past year
<3> About once a month
<4> About once a week

Problems:

This question was well understood. A couple of respondents gave responses (twice a month; twice a week.) that they did not know how to fit into the response categories provided on the flashcard. Item was not retested in Round 2.

Recommendation:

Revise response options so <3> is "about one or twice a month "and <4> is "once a week or more."

Round 1
Tested wording:

About how many children's book does (child) have in your home now, including library books?

(Universe: Ages 3-11)
___ number of books

Problems:

This question did not work well. Most respondents reported large numbers of books: 25, 30, 50, 60, 100, etc.. It was clear that reports such as these were wild guesses (15 of the 18 respondents included in Round 1 were low income and even they, for the most part, were providing reports of large numbers of books.) We asked CTI if there were specific ranges of numbers of books that are important analytically. For example, are outcomes of children that have 25 books different from those that have 60 or 100 books? Or is it important only to know if the child has less than 5, 6-10... or 30+. CTI concurred that ranges can be used, instead of obtaining an exact number of books. The item was not retested in Round 2.

Recommendation:

Revise response format so ranges are used instead of recording a specific number of books. Based upon CTI's suggestion, we recommend the ranges used be those from the NLSY HOME scale, as shown below:

None
1 or 2 books
3 to 9 books
10 to 19
20 or more books.

Round 1
Tested wording:

How often do you or any family member go with (child) somewhere--like out to the movies, to the park, to a sports event, or to a shopping mall?

(Universe: Ages 9-17; Flashcard)

<1> never
<2> several times a year
<3> about once a month
<4> about once a week
<5> everyday or almost everyday

Problems:

The question seemed well understood. A variety of activities were captured in response to this question. Item was not retested in Round 2.

Round 1
Tested wording:

How often does (child) visit a museum, such as a scientific, art, historical, or children's museum?

(Universe: Ages 9-17; Flashcard)

<1> never
<2> several times a year
<3> about once a month
<4> about once a week
<5> everyday or almost everyday

Problems:

Question seems well understood. Many respondents mentioned school field trips in response to this item. Item was not retested in Round 2.

Round 1
Tested wording:

About how often does (child) read?

(Universe: Ages 9-17; Flashcard)

<1> never
<2> several times a year
<3> about once a month
<4> about once a week
<5> everyday or almost everyday

Problems:

Most respondents included reading for pleasure and for school. A couple of respondents said they would have had a very different answer if the question was only asking about reading for pleasure. We requested additional information from CTI regarding the intent of the question. CTI informed us that the question is ONLY suppose to capture reading for pleasure and suggested question wording used in the NLSY HOME which is "About how often does (child) read for enjoyment? " Item was not tested in Round 2.

Recommendation:

About how often does (child) read for enjoyment?

Round 1
Tested wording:

How often does (child) go to a religious service, a religious social event, or to religious education such as Sunday school?

(Universe: Ages 9-17; Flashcard)

<1> never
<2> several times a year
<3> about once a month
<4> about once a week
<5> everyday or almost everyday

Problems:

The question worked fine. However, response options caused some difficulty for a few respondents. In those cases, respondents reported "twice a week ", but could not find an appropriate response option. Item was not retested in Round 2.

Recommendation:

Revise response options so <3> reads "about once or twice a month and <4> reads "once or twice a week. "

Round 1
Tested wording:

About how often does (child) go to the library to read or check out books? (Universe 12-17; Flashcard)

<1> never
<2> several times a year
<3> about once a month
<4> about once a week
<5> everyday or almost everyday

Problems:

The question worked fine. Appropriately, respondents were not including visits to the school library. Item was not retested in Round 2.

Round 1
Tested wording:

How often has (child) gone on trip an hour or more from home? (Universe : Ages 12-17; Flashcard)

<1> never
<2> several times a year
<3> about once a month
<4> about once a week
<5> everyday or almost everyday

During the past year, did (child) attend an overnight or day camp for a week or more? (Universe : Ages 9-17)

<1> Yes
<2> No

During the past year, did (child) attend an overnight trip or retreat sponsored by a school, religious, or community group? (Universe : Ages 9-17)

<1> Yes
<2> No

During the past year, did (child) go on an overnight trip with family or friends? (Universe : Ages 9-17)

<1> Yes
<2> No

Problems:

This series of questions (asked of 5 respondents) seemed overly burdensome. Many respondents did not seem to hear the phrase "an hour or more "in the first question. Some defined "a trip"as going to the grocery store, going to a friend's house, going to the movies etc. For the last question, in response to probing, many respondents said they would count as an overnight trip, times when the child sleeps at a friends house, even if it's only around the corner. We requested additional information from CTI regarding the definition of trip and the objectives of the question series. To reduce respondent burden, we asked CTI for the two most important concepts regarding trips and requested that only two questions regarding trips be asked.

Recommendation:

Replace series with the following two items and retest.

During the past year, how often has (child) gone on a trip at least an hour or more away from home?

During the past year, did (child) go on an overnight trip or retreat with family or friends, or attend an overnight camp?

CTI recommended that FRs be trained that this could include a trip or camp sponsored by a school, religious, or community group.

Round 2
Tested wording:

During the past year, how often has (child) gone on a trip at least an hour or more away from home? (Universe: Child is 9-11; Flashcard)

<1> Never
<2> Several times a year
<3> About once a month
<4> About once a week
<5> Everyday or almost everyday

Problems:

At least three respondents answered the question as to how often the child has gone on a trip without a parent. They associated "away from home"with "away from a parent." Two respondents attempted to clarify whether or not the parent was on the trip with the child. They both came to the conclusion that the parent could be on the trip. Lesser problems included whether or not school trips, or visits to relative homes were included. One respondent did include school trips that were probably not 1 hour or more away from home (e.g., bowling). Some respondents only included overnight trips; other respondents included day trips as well. Sleep overs were not included in the counts provided. "Trips"were typically defined as out of the neighborhood or out of the metropolitan area. The addition of a reference period of "past year"to the first question was helpful since in Round 1, we did not include a reference period and that led to some confusion for respondents.

Recommendation:

Revise wording of first question to read: During the past year, how often has (CHILD) gone on a trip at least an hour or more from home?

Round 2
Tested wording:

During the past year, did (child) go on an overnight trip or retreat with family or friends, or attend an overnight camp? (Universe : Child is 9-11)

Problem:

Again, it is not apparent to respondents that the question is asking about overnight trips including the parent. (This may be a carryover effect from interpretation of the previous question.) Three respondents included only overnight trips away from the parent. One respondent answered based on a day trip; not an overnight trip.

Recommendation:

A suggested revision: During the past year, did (child) go on an overnight trip or retreat with you, with family or friends, or attend an overnight camp?

Series Comment:

If this series is adopted for SPD99, keep in mind that only a subset of the series was retested in Round 2. Be sure to include the other items from Round 1 that were not revised.

C. Enrichment Activities

(Universe: Ages 6-17)

We tested the enrichment questions during both rounds of testing. We reordered the series in Round 2 and made a few minor wording changes. Part way through the Round 2 testing, we received word from CTI and Population Division that the wording for these items was under discussion. CTI preferred we use wording from the 1992 SIPP Wave 9 Child Well-being module, instead of SPD 98 wording or revised wording. Based upon instructions from the Population Division, the Round 2 wording was modified and is labeled as Round 2, Version 2 below. The following discussion is organized by the question topic. We will discuss the results from both rounds of testing for each topic included in the enrichment series (e.g. participation in sports teams, lessons, clubs or organizations, etc.). This is followed by the results of the questions on frequency of participation in these activities during the previous year.

1. Sports teams

Round 1
Tested wording:

Since March 1997, was [child] on any kind of organized sports team?

Problems:

No problems identified.

Round 2 Version 1
Tested wording:

The next few questions are about activities (child) may have participated in outside of the regular school day.

Since March 1997, was [child] on any kind of organized sports team?

Problems:

No problems identified. Three respondents understood this question to be league teams, either through the school, municipality, or neighborhood.

Round 2 Version 2
Tested wording:

Since April 1997, was [child] on any kind of a sports team?

Problems:

No major problems identified. One respondent included participation in physical education classes as an affirmative answer to this question. One respondent included karate as a sport and answered this question affirmatively as well as the subsequent question on "lessons"because that question specifically mentioned "karate"in the list of examples (double counting of activity). Another respondent mentioned dance lessons in middle school, but determined that the activity happened prior to the reference date.

2. Lessons

Round 1
Tested wording:

Did [child] take lessons after school or on weekends in activities such as music, dance, language, or karate at any time since March 1997?

Problems:

All but three respondents reported "no"to this question. Respondents understood the term "lessons,"but may have been thrown off by the term "activities"in the question. Some respondents thought the question was asking about additional activities, not lessons. Some respondents thought the question was asking specifically about "music, dance, language, or karate,"rather than interpreting these as examples.

Recommendation:

We recommend revising the wording so that the activities listed will be interpreted as examples rather than as an exhaustive list and changing the word "activities"to "subjects"as shown below:

"Since March 1997, did (child's name) take lessons after school or on weekends in music, dance, language, karate, or any other subject?"

Round 2, Version 1
Tested wording:

Since March 1997, did (child) take lessons after school or on weekends in music, dance, language, karate, or any other subject?

Problems:

The addition of "or any other subject"is confusing since karate is considered a sport to some people. One respondent started to include lessons received while on organized sports teams. One respondent reported several individual activities including swimming and karate. When probed about whether they interpreted the list to be an exhaustive list of what was to be included or if it was just a list of examples, most respondents said it was an exhaustive list.

Round 2, Version 2
Tested wording:

Did (child) take lessons after school or on weekends in activities such as music, dance, language or karate, at any time since April 1997?

Problems:

Question seemed to be well understood. Karate is considered to be a team sport for some respondents; thus, it was double counted under team sports and under lessons. This is the same question that was tested during Round 1. Differences in results may be due to the respondent pool.

3. Clubs/Organizations

We tested several different versions of questions on participation in clubs and organizations. During all of Round 1 and during Version 1 of Round 2, we asked separate questions on participation in 1) religiously-affiliated groups, 2) school-based clubs, and 3) civic or charitable clubs. Based on discussions between the Census Bureau's Population Division and CTI regarding time series concerns, we used the same question wording used in the 1998 SPD core questionnaire during Version 2 of Round 2. This is a single question asking about participation in any club or organization after school or on weekends. To capture detailed information on the specific type of club or organization, we asked three follow-up questions for persons responding positively to the 1998 SPD item, in order to determine if the activity was religiously-affiliated, school-based, or civic/charitable in nature.

In order to understand the differences among the three versions tested, we present the questions on clubs/organizations for the three questionnaire versions tested below. For ease of presentation, the frequency questions that would follow each specific club/organization question have been omitted as are the other enrichment questions that were interspersed between the "club/organization"questions in the questionnaires tested.

Round 1
  1. The next few questions are about activities (child) may have participated in outside of the regular school day.

    2. Since March 1997, did [child] participate in any religiously-affiliated youth groups or organizations, not including attendance at regular religious services?

  2. Did (child) participate in any school-based clubs or organizations after school or on weekends, such as band, chorus, drama club, or student government since March 1997?

  3. Did (child) participate in any other civic or charitable organizations such as Scouts, Big Brothers or Sisters, 4H, or any similar organizations since March 1997?

Round 2, Version 1
  1. Since March 1997, did (child) participate in any civic or charitable organization, such as Scouts, Big Brothers or Sisters, 4H, or any similar organization?

  2. Since March 1997, did (child) participate in any school-based clubs or organizations such as band, chorus, drama club, or student government?

  3. Since March 1997, did (child) participate in any religiously-affiliated activities, youth groups, or organizations, including attendance at religious services?

Round 2, Version 2
  1. Did (child) participate in any clubs or organizations after school or on weekends, such as Scouts, school newspaper, Boys/Girls club, or a religious group at any time since April 1997? (This is a "screener"item. If "yes,"then we ask items 2 through 4 below.)

  2. Were any of them school-based clubs or organizations such as band, chorus, drama club, or student government?

  3. Of these clubs or organizations that (child) participated in, were any of them civic or charitable organizations such as Scouts, Big Brothers or Sisters, 4H, or any similar organization?

  4. Of the clubs and organizations that (child) participated in, were any of them religiously-affiliated activities, youth groups or organizations, including attendance at religious services?

Below are the item-by-item results for each round of testing. Shown first are the results for the "screener"item from Version 2 of Round 2. Following that are the results for each round of testing by the type of club/organization.

Round 2, Version 2
Tested wording:

Did (child) participate in any clubs or organizations after school or on weekends, such as Scouts, school newspaper, (Boys/Girls) club, or a religious group at any time since April 1997? (If "yes,"ask the follow-up questions)

Problems:

Although we have no quantitative data, this collapsed question might underestimate the amount of activities as compared to the earlier versions tested in which this question was decomposed into three separate items. Cognitive testing indicates that many respondents think the lists provided in questions are exhaustive rather than exemplary. The decomposed questions list additional activities that might not come to mind when answering this question.

There also needs to be a clearer definition of religious clubs or organizations. Two respondents mentioned religious education classes. One of these respondents included religious education classes by answering affirmatively to this question; the other respondent did not. That respondent did not include it because s/he did not consider it to be a group. She said the child didn't sign up for "Sunday school like you would sign up for Boy Scouts; we just participate."

 a. Religiously-affiliated groups

Round 1
Tested wording:

The next few questions are about activities (child) may have participated in outside of the regular school day.

Since March 1997, did [child] participate in any religiously-affiliated youth groups or organizations, not including attendance at regular religious services?

Problems:

This question was not well understood. Some respondents who responded positively to the question, reported activities that were not associated with the church, such as sports teams. Perhaps the adjective "religiously-affiliated" didn't carry over to the nouns "groups" and "organizations," since some respondents reported groups/organizations that their children belonged to even though they were not associated with a religious group. Other respondents answered positively because their children are on sports teams sponsored by the church or in a Boy Scout organization that is affiliated with the church. Other respondents were unclear whether to report church-related activities, such as being an usher boy, since the duty was done during regular church service. One respondent didn't think her child's participation in Bible class should be included even though the activity occurred during the summer. Another respondent reported "no" and then changed her answer to ask if her daughter's attendance at confirmation class would count.

Based on cognitive testing, we think that the order of the Round 1 enrichment questions is awkward and that it would be better not to start the series with a religiously-oriented question. In addition, there is a separate question on attendance at religious services, social events and classes in the cognitive stimulation section, and we wondered whether the two questions could be combined into one item on religious activity. In Round 2, we reordered the series and combined the two items.

Round 2, Version 1
Tested wording:

Since March 1997, did [child] participate in any religiously-affiliated activities, youth groups, or organizations, including attendance at religious services?

Problems:

The inclusion of attending religious services with the other religious-affiliated activities is problematic. Respondents do mention attendance at religious services. When children participated in both religious youth groups and religious services, respondents did not always report the frequency of participation accurately in subsequent frequency questions. When asked frequency, they often answered once a week or four times a month, even though there were other activities mentioned besides attendance at services once a week. Respondents would have an easier time reporting attendance at religious services separately from other religious activities. Also, it is not clear if the question is asking only about other religious youth groups (i.e., not classes) or if it is asking, as one respondent paraphrased, "Is he actively participating in a church group?"which could be a religious class.

Round 2, Version 2
Tested wording:

Of the clubs and organizations that (child) participated in, were any of them religiously-affiliated activities, youth groups, or organizations, including attendance at religious services?

Problems:

Including attendance at religious service is a problem in this question. Attendance should be captured in another question. Also, the types of religiously-affiliated activities that are to be included need to be clarified.

Problems similar to those discussed in the Version 1 wording were encountered when this question was asked. It confirmed the need for two separate questions to determine participation in religiously-affiliated activities such as youth groups and organizations and to determine participation in religious services.

b. School-based clubs/organizations

Round 1
Tested wording:

Did [child] participate in any school-based clubs or organizations after school or on weekends, such as band, chorus, drama club, or student government, since March 1997?

Problems:

Respondents understood that the question is asking about "school-based clubs or organizations." However, some respondents did not focus on the "after school or on weekends" part of the question and reported clubs that children participate in during the school day. When asked to paraphrase, some respondents thought that the question included activities "sponsored by the school,... during school or after school."One respondent reported that she would include band, because "it's like after school....where you have to practice." Other respondents clearly heard the "after school or on weekends" and reported that their children participate in, for example, band or student government, but that it meets during school hours. Some respondents thought that we were asking only about the specific activities included in the list of examples.

Round 2, Version 1
Tested wording:

Since March 1997, did [child] participate in any school-based clubs or organizations such as band, chorus, drama club, or student government?

Problems:

Many of the respondents said they thought the question was asking only about the activities listed. We did not receive enough information about whether the activities mentioned were during or after school and are unclear whether it is important analytically that the activities occur after school or on weekends, since in some schools students participate in these activities during the regular school day.

Round 2, Version 2
Tested wording:

Were any of them school-based clubs or organizations such as band, chorus, drama club, or student government?

Problems:

This question does not specify that these activities are after school. Many of the activities listed could occur during school hours. Only two respondents were asked this question. Both of them answered no. Since one of the groups, a teen group was associated with the Catholic school, we were surprised that this respondent didn't answer affirmatively.

c. Civic or charitable clubs/organizations

Round 1
Tested wording:

Did [child] participate in any other civic or charitable organization, such as Scouts, Big Brothers or Sisters, 4H, or any similar organizations since March 1997?

Problems:

No problems identified, but few respondents reported that their children participated in such activities.

Round 2, Version 1
Tested wording:

Since March 1997, did [child] participate in any civic or charitable organization, such as Scouts, Big Brothers or Sisters, 4H, or any similar organizations?

Problems:

No major problems identified.

Round 2, Version 2
Tested wording:

Of these clubs or organization that (child) participated in, were any of them civic or charitable organizations, such as Scouts, Big Brothers or Sisters, 4H, or any similar organization?

Problems:

No problems identified.

d. Months Active: Round 1
These questions were asked if the respondent answered affirmatively to an enrichment question.

Tested wording:

In the last year, how many months did (child) participate in these activities?

<1> 1-2 months
<2> 3-6 months
<3> 7-10 months
<4> 11-12 months

Problems:

One respondent asked if the reference period was the calendar year.

Recommendation:

Reword the question: "How many months did (child) participate in these activities since (date)?"

Tested wording:

In the months when [child] was participating in these activities, was this once or twice a month, about once a week, or several times a week?

<1> Once or twice a month
<2> About once a week
<3< Several times a week

Problems:

No problems identified other than team practice time is not always included. Also, theoretically it could be confusing if frequencies differ for multiple activities being reported in the same question. Also, some respondents were looking for a response option of "twice a week."

Recommendation:

Revise response option <2> to read "about once or twice a week."

e. Months Active: Round 2, Version 1
These questions were asked if the respondent answered affirmatively to an enrichment question.

Tested wording:

How many months did (child) participate in these activities since March 1997? (Universe: )
<1> 1-2 months
<2> 3-6 months
<3> 7-10 months
<4> 11-12 months

Problems:

No problems identified. Only one respondent received this version.

Tested wording:

In what months did (child) participate in these activities since April 1997?

__April 1997              __ October 1997
__May 1997               __ November 1997
__June 1997               __ December 1997
__July 1997                __ January 1998
__August 1997           __ February 1998
__ September 1997    __ March 1998

Problems:

No problems identified. Most respondents could answer this question, Respondents typically used the reference periods of the start and stop times of the different sports seasons to help benchmark their answers. Typically respondents did not have difficulty in reporting the actual month although at least one respondent reported all months and then qualified that answer with an except statement. When there was more than one activity, one respondent reported separately for each activity, e.g., "She's been in ice skating for a year since Feb 97, she's been in swimming during the Fall and the Winter."Months included September 97 through March 98. Karate she started in Jan 98 ..."

f. Frequency Questions: Rounds 1 and 2
Tested wording: In the months when [child] was participating in these activities, was this once or twice a month, about once or twice a week, or several times a week?

<1> Once or twice a month
<2> About once or twice a week
<3< Several times a week

Problems:

No problems identified.

4. Gang-related activity

Round 1
Tested wording:

Is (child) a member of a gang that gets into trouble with the law?

Problems:

Question was well understood.

Round 1
Tested wording:

Has (child) ever gotten into trouble with the law because of this gang?

Problems:

No children reported as gang member, so the question was never asked.

Round 1
Tested wording:

How many times has [child] gotten into trouble with the law because of this gang?

Problem:

No children reported as gang member, so the question was never asked.

Round 1
Tested wording:

How old was [child] when [he/she] [first] got into trouble with the law because of this gang?

Problems:

No children reported as gang member, so the question was never asked.

Round 1
Tested wording:

Has [child] ever been arrested?

Problems:

Question was well understood.

Round 1
Tested wording:

Did that happen once or more than once?

Problems:

No children were reported as previously arrested so the question was never asked.

D. Positive Behaviors/Social Competence

(Universe : Ages 3-11)

Round 2
Tested wording:

Here are some statements which may or may not describe your child. As I read each statement, decide whether it is often true, sometimes true, or not true at all of your child over the last three months. (Fill name of child):

a. Is warm and loving. Is that often true, sometimes true, or not true at all of (fill name)? (Same response options used for all items)


<1> Often true
<2> Sometimes true
<3> Not true at all

b. Gets along well with other children. Is that often true, sometimes true, or not true at all of (fill name)?

c. Is admired and well liked by other children. Is that often true, sometimes true, or not true at all of (fill name)?

d. Shows concern for other people's feelings. Is that often true, sometimes true, or not true at all of (fill name)?

e. Is helpful and cooperative. Is that often true, sometimes true, or not true at all of (fill name)?

f. Is considerate and thoughtful of other children. Is that often true, sometimes true, or not true at all of (fill name)?

g. Tends to give, lend, and share. Is that often true, sometimes true, or not true at all of (fill name)?

Problems Series:

The series of questions seem to be well understood .

During our initial review of this series, prior to testing, we modified the last response option (within the question) to read "not true at all "instead of "not true." This was done so the response options would be consistent with the wording of the introduction. During cognitive testing, none of the respondents chose the "not true at all" response option. We wonder if that may be due, in part, to the modified wording which, in retrospect, sounds so definitive that it is unlikely any parent would select that option.

Also, during the first three interviews, the question wording did not specify the name of the reference child. For interviews 4-16, we used a slightly revised wording that included the child's name so the parent could stay focused on which child they were providing information.

Recommendation:

We recommend the following question wording/response options:

Here are some statements which may or may not describe your child. As I read each statement, decide whether it is often true, sometimes true, or not true of your child over the last three months. (Fill name of child):

Is warm and loving. Is that often true, sometimes true, or not true of (fill name)?
<1> Often true
<2> Sometimes true
<3> Not true

(Universe : Ages 12-17)

Tested wording:

Here are some statements which may or may not describe your child. As I read each statement, decide whether it is often true, sometimes true, or not true at all of your child over the last three months. (Fill name):

a. Plans ahead. Is that often true, sometimes true, or not true at all of (fill name)?

b. Is admired and well liked by kids the same age. Is that often true, sometimes true, or not true at all of (fill name)?

c. Looks for ways to help other people. Is that often true, sometimes true, or not true of at all of (fill name)?

d. Is considerate and thoughtful of other people. Is that often true, sometimes true, or not true at all of (fill name)?

e. Goes out of his or her way to make the family feel proud. Is that often true, sometimes true, or not true at all of (fill name)?

f. Is able to express his/her thoughts and feelings to other people. Is that often true, sometimes true, or not true at all of (fill name)?

g. Is warm and affectionate toward members of our family. Is that often true, sometimes true, or not true at all of (fill name)?

h. Spends his or her own time helping other people. Is that often true, sometimes true, or not true at all of (fill name)?

Problems (Series):

No significant problems were detected with this series of questions. However, it should be noted that items c and h are very similar concepts and consideration should be given to deleting one or the other, in an effort to reduce respondent burden (if it can be done without compromising the integrity of the scale). Additionally, the recommendations for the response options in the prior series apply here as well.

E. Grades/Achievement
(Universe : ages 6-17)

Tested Wording:

Based on your knowledge of (child's) schoolwork, including (his/her) report cards, how has (he/she) been doing in school overall? Would you say very well, well, average, below average or not well at all?

<1> Very well
<2> Well
<3> Average
<4> Below average
<5> Not well at all
<6> Child not attending school

Problems:

Question was well understood. Most respondents indicated they thought about multiple sources of information regarding child's schoolwork, not just report cards.

F. Household Resources (aka Cognitive Stimulation)
(Universe: Ages 0-17 )

Round 1
Tested wording:

Does your home have a computer?

Problems:

This question was well understood, although one respondent questioned whether children's computers like "Vtech," which contains learning activities, should be included. This should be addressed in the FR manual. This was not tested in Round 2.

Round 1
Tested wording:

Does your home have a quiet place to study?

Problems:

Respondents interpreted the question as asking about a quiet place to study somewhere inside the house, but not necessarily at a desk or other designated spot that is exclusively used for studying. All respondents reported having a quiet place to study in their households, even large families living in relatively small living spaces. This is another item for which the utility of the data may be questioned. If everyone reports "yes ", how valuable is the data? Do we know from other sources if there is any variance in responses to this item? This item was not tested in Round 2.

Round 1
Tested wording:

Does your home have a dictionary?

Problems:

This question is clearly understood. All respondents reported having a dictionary. Do we know from other sources if there is any variance in responses to this item? This item was not tested in Round 2.

Round 1
Tested wording:

About how many children's records, audio tapes, CD's, or music videos do you have at home for (fill names of children under age 12), including any from the library?

Problems:

This question does not work. We estimate that, at most, only 5 of the 17 respondents answered this question with a count of music-related records, audio tapes, CDs or videos. More typically this question captured a count of VCR videos such as Disney or other movies, in addition to cassette tapes. Also, four respondents mentioned that their audiotape cassettes included stories (religious and otherwise) or learning activities rather than just music. We requested additional information from CTI regarding the objective of the question. CTI informed us that it is to learn about cognitively stimulating materials resources that parents provide for their children. And, CTI stated that children's movie videos can be included in responses to this item. CTI recommended rewording this question.

Recommendation:

We recommend deleting this item. CTI recommends rewording the question to read: "About how many children's records, audiotapes, CDs, or children's videos do you have at home, including any from the library? They also recommend moving this item to immediately after the "number of books" item in the Cognitive Stimulation section. If the question is retained, then we recommend using response options of "ranges" (as recommended for the "number of books" item), instead of asking for an exact number.

Series Comment:

Given the problem with the last item, as well as the lack of variance in responses to the second two items, this series should probably be a low priority for inclusion in the SPD99.

G. Conflict Between "Parents" Who Co-reside
(Universe is two-parent households, which includes biological, step, adoptive, and foster parents, and households in which one parent is living with an unmarried partner, ages 0-17, flashcard)

Round 1
Tested wording:

The following is a list of subjects on which couples often have disagreements. How often, if at all, in the past year have you had open disagreements about each of the following:

a. Household tasks (same response options used in all items)

<1> Never
<2> A few times a year
<3> Once a month
<4> Several times a month
<5> About once a week
<6> Several times a week
<7> Almost every day

b. Money

c. Spending time together

d. Sex

e. In-laws

f. The children

Problems:

The term "open disagreements" was understood differently by different respondents. (Some respondents even asked the researchers what was meant by the term.) In some cases, it meant that the argument is heard by people other than the two having the disagreement ("open for others to hear"). In other cases, it meant that the argument was being discussed "openly," (as opposed to being held inside and not discussed) but didn't necessarily have to be heard by others. We requested additional information from CTI regarding how "open disagreements" are defined. CTI recommended dropping the term "open" to eliminate confusion. This series was not tested in Round 2.

Recommendation:

The following is a list of subjects on which couples often have disagreements. How often, if at all, in the past year have you had disagreements about each of the following:

a. Household tasks (same response options used in all items)

<1> Never
<2> A few times a year
<3> Once a month
<4> Several times a month
<5> About once a week
<6> Several times a week
<7> Almost every day

b. Money

c. Spending time together

d. Sex

e. In-laws

f. The children

Round 1
Tested wording:

Sometimes arguments between partners become physical. During the last year, has this happened in arguments between you and your spouse (partner)?

Problems:

This question was well understood. It was not tested in Round 2

H. Conflict Between Biological Parents Who Do Not Co-reside
(Universe : Households where there is an absent biological or adoptive parent of one of the resident children, ages 0-17, flashcard)

Round 1
Tested Wording:

I am going to read you a list of issues that you and (child's) (mother/father) may have conflict over. For each one, please tell me if you have no conflict, a little, some, pretty much, or a great deal of conflict. How much conflict do you have about....

a. where child lives? (Same response options used for other items)

<1> No conflict
<2> A little conflict
<3> Some conflict
<4> Pretty much conflict
<5> Great deal of conflict
<6> I have no contact with my child's biological (mother/father)

b. how (he/she) is raised?

c. how you spend money on (child)?

d. how (he/she) spends money on (child)?

e. the time (she/he) spends with (child)?

f. (her/his) financial contribution to (child's) support?

Problems:

Some custodial parents never have contact with the non-custodial parent. Some of the respondents reported "no conflict" instead of using the "I have no contact with my child's biological (mother/father)" response option. Perhaps a screening item is needed. Additionally, there is no reference period in the question, which presents some confusion (although you could assume it is "current" due to verb tense within the questions). We asked CTI what timeframe was relevant for this set of questions and we also asked them how conflict is defined for this series. CTI informed us that conflict is defined as "disagreements, arguing, or fighting." CTI suggested a timeframe of "the past year." They agreed with our suggestion to screen on last contact.

Recommendation:

We included a reference period of "the past year" to increase the likelihood that respondents will use the same reference period when answering the questions. We included a question on "date of last contact" to determine whether the series should be asked. Only persons who had contact in the last year were asked this series during Round 2. We also included the definition of conflict within the question and included the phrase "you and child's (father/mother)" so it would be clear whose conflict we were asking about.

Revised wording follows:

In what month and year did (designated parent) last have contact of any kind, including phone calls, letters, or face-to-face contact with (child's) (mother/father)?

____Month ______Year (If prior to March 1997, skip to next series.)

I am going to read you a list of issues that you and (child's) (mother/father) may have conflict over. By conflict we mean, arguments, disagreements, or fighting. For each one, please tell me if you have no conflict, a little, some, pretty much, or a great deal of conflict.

This series was tested in Round 2.

Round 1
Tested wording:

Sometimes arguments between former partners become physical. During the last year, has this happened in arguments between you and (child's) (father/mother)? (Universe: Ages )

Problems:

Question was well understood . Question was not tested in Round 2.

Round 2
Tested wording:

In what month and year did (designated parent) last have contact of any kind, including phone calls, letters, or face-to-face contact with (child's) (mother/father)?

____Month ______Year (If prior to March 1997, skip to next series.)

I am going to read you a list of issues that you and (child's) (mother/father) may have conflict over. By conflict we mean, arguments, disagreements, or fighting. For each one, please tell me if you have no conflict, a little, some, pretty much , or a great deal of conflict.

During the past year, how much conflict did you and (child's father/mother) have about....

a. where child lives?
<1> No conflict
<2> A little conflict
<3> Some conflict
<4> Pretty much conflict
<5> Great deal of conflict
<6> I have no contact with my child's biological (mother/father)

b. how (he/she) is raised?
c. how you spend money on (child)?
d. how (he/she) spends money on (child)?
e. the time (she/he) spends with (child)?
f. (her/his) financial contribution to (child's) support?

Problems (Series):

(Only 4 respondents went through this series of questions.) For the most part, this series worked fine and there were no problems with the individual items.

In this series, there are two items (d and f) related to financial spending on/for the child by the absent parent. In some cases, respondents interpreted item d as asking about child support payments, which we don't think is the intent of the question. It seems unnecessary to include two such closely related items.

Recommendation (Series):

To ease respondent burden, we recommend deleting item d (how he/she spends money on child) if the integrity of the scale will not be affected.

Series Comment:

If this series is adopted for SPD99, keep in mind that the physical abuse question tested in Round 1 also goes with this series of questions.

I. FAMILY CONFLICT
(Universe: Ages 0-17)

Round 1
Tested wording:

Next are some statements about how families get along and settle arguments. For each one please indicate if you strongly agree, somewhat agree, somewhat disagree, or strongly disagree that this statement is like your family.

Problems:

For this series of questions, the term "family" had multiple definitions. Several respondents considered their "family" to include extended family members not currently living with the respondents. Other respondents considered their "family" to include only those relatives with whom they were currently living at the time of the interview. Similarly, respondents had a variety of timeframes for which they answered the questions. It appeared to be divided among the current time period, a few years back, an entire lifetime and particular time period.

We requested additional information from CTI on the issues identified above. CTI informed us that for this set of questions "family" is defined as the family that lives in the household with the respondent. CTI preferred that we not modify the lead-in to define what was meant by "family."

Recommendation:

Modify the lead-in to define "family." The recommended wording follows:

Next are some statements about how families get along and settle arguments. For each one please indicate how strongly you agree or disagree that the statement is like the family you currently live with.

Round 2
Tested wording:

Next are some statements about how families get along and settle arguments. For each one please indicate how strongly you agree or disagree that the statement is like the family you currently live with.

Problems:

The addition of the phrase "the family you currently live with" to the lead-in helps distinguish what family we are referencing. One respondent paraphrased "the family that we currently live with." One respondent paraphrased, "immediate family." One respondent listed all people in her immediate family that she lives with.

Recommendation:

Retain Round 2 wording for lead-in that defines "family."

Round 1
Tested wording:

We fight a lot in our family. Do you strongly agree, somewhat agree, somewhat disagree, or strongly disagree?

<1> Strongly agree
<2> Somewhat agree
<3> Somewhat disagree
<4> Strongly disagree

Problems:

Respondents included both verbal and physical fighting. We requested more information from CTI regarding the types of fighting that should be included. CTI responded that fighting can include verbal and physical fighting. CTI did not think that "discussions about different points of view" should be included as fighting unless these discussions escalated into what a respondent would call a "fight."

Recommendation:

Retain question wording and retest in Round 2.

Round 2
Tested wording

We fight a lot in our family. Do you strongly agree, somewhat agree, somewhat disagree, or strongly disagree?

Problems:

No problem identified. Respondents included both verbal and physical fighting. Only a few respondents included one and not the other.

Round 1
Tested wording

Family members hardly ever lose their tempers. Do you strongly agree, somewhat agree, somewhat disagree, or strongly disagree?

Problem:

Some respondents seem to interpret this question as a double negative and were confused. Perhaps using a term other than "hardly ever" would be better. Consideration should be given to "Family members rarely lose their tempers."

Recommendation:

We recommend retesting this item in Round 2.

Round 2
Tested wording

Family members hardly ever lose their tempers. Do you strongly agree, somewhat agree, somewhat disagree, or strongly disagree?

Problems:

It appears that many of the respondents (at least 5) were confused by this question. In combination with the response categories, it is a double negative. Two respondents actually indicated that it was a double negative. Two of the four respondents, who clearly understood this question, asked if the word "seldom" was used in the question. In addition, at least two respondents had difficulty with the term "family members" and starting talking about their extended family that did not live in their household. When probed, one respondent said that the first question that uses "our family" sounded more narrowly defined than the later questions that used the term "family members. (Keep in mind however, that, in general, most respondents had a clear definition of "family members" that was consistent with the intended definition.) It could also be that our probing on the definition of "family members" led these two respondents to this misinterpretation since the lead-in (which defines family members) is then distanced from the detailed questions.

Recommendation:

If this series of questions is used in SPD99, we recommend that this item be reworded and offer a few alternative wordings, in the order of our preference:

1. Family members lose their tempers. "This eliminates the modifier of "hardly ever." (Eliminating the modifier on all the questions might prove to provide more consistent answers across the series. This recommendation is based more on an expert review, than on results of cognitive interviewing.)

2. "Family members often lose their tempers." This replaces "hardly ever" with "often." This changes the direction of the statement from a negative direction to a positive direction. Since the other statements are this direction, this maintains consistency among the questions.

3. "Family members seldom lose their tempers. This replaces "hardly ever" with "seldom." This maintains a similar direction to the question and response in the original version. "Seldom" was used by two respondents when paraphrasing the original question wording.

Round 1
Tested wording:

Family members sometimes get so angry they throw things. Do you strongly agree, somewhat agree, somewhat disagree, or strongly disagree?

Problems:

No problems were identified. Item not tested in Round 2.

Round 1
Tested wording:

Family members always calmly discuss problems. Do you strongly agree, somewhat agree, somewhat disagree, or strongly disagree?

Problems:

No problems were identified. Item not tested in Round 2.

Round 1
Tested wording:

Family members often criticize each other. Do you strongly agree, somewhat agree, somewhat disagree, or strongly disagree?

Problems:

No problems were identified. Item not tested in Round 2.

Round 1
Tested wording:

Family members sometimes hit each other. Do you strongly agree, somewhat agree, somewhat disagree, or strongly disagree?

Problems:

Some respondents included horsing around and playful hitting when answering this question. We requested additional information from CTI about what was suppose to be included/excluded when responding to this question. CTI responded that they would not include playful hitting in this item. This could include children hitting children, but it would be hitting in an angry or aggressive manner, not playful hitting. We suggested clarifying what should be included by adding the term "in anger" to the question.

Recommendation:

Family members sometimes hit each other in anger.

Round 2:
Tested wording:

Family members sometimes hit each other in anger. Do you strongly agree, somewhat agree, somewhat disagree, or strongly disagree?

Problems:

No problems identified. The addition of the term "in anger" helped define what level of hitting we are interested in. The word "violence" was used by three respondents when expounding on their answers. Also, respondents used the term "in anger" within the explanation. For example, "Because we never hit each other in anger." Most respondents did not include playful hitting or horsing around in their answer. A few respondents did include children hitting each other, as in "the kids hit each other." This seemed to be when the children were mad and not when they were playful.

Recommendation:

Retain Round 2 question wording.

Problems (Series):

From the cognitive interviews there were no apparent problems with the response options. We feel however that this entire series could be posed as a series of questions asking for frequencies, instead of agree/disagree statements. The response categories could then be more of a frequency scale. We felt that there wasn't always a large distinction between the meaning of sometimes agree and sometimes disagree.

Series Comment:

If this series is adopted for SPD99, keep in mind that only a subset of the series was retested in Round 2. Be sure to include the other items from Round 1 that were not revised.

J. Attitudes Towards Welfare/work
(Universe : All respondents)

Round 1:
Tested wording:

Here are some opinions that people have expressed about welfare. We are interested in knowing how much you agree or disagree with each opinion.

Welfare makes people work less than they would if there wasn't a welfare system. Do you strongly agree, agree, you're in the middle, disagree or strongly disagree?

<1> strongly agree
<2> agree
<3> in the middle
<4> disagree
<5> strongly disagree
<d> don't know

Problems:

The statement "makes people work less" is confusing to respondents. It seems some respondents thought this question was asking if people on welfare were lazy. Others mistook "welfare" as a person, per chance a case worker and based their answer on the fact that previously, welfare didn't allow you to work, lest you lose your benefits.

Recommendation:

Revise question wording and retest in Round 2. (We were unable to develop question wording with which we were satisfied so we retested the original wording again in Round 2.)

Round 2:
Tested wording:

Welfare makes people work less than they would if there wasn't a welfare system. Do you strongly agree, agree, you're in the middle, disagree or strongly disagree?

Problems:

During the second round of testing, this statement was understood more clearly, which may be due to differences in the respondent pool. Typically, respondents added the word "system" to the word "welfare" and talked about the "welfare system" as evidenced in the following quotes: "If there wasn't a welfare system, then they would work more." or "If there wasn't a welfare system, they would be more actively seeking work, out there trying to find jobs and stuff." or "Because there is welfare, some people...if there weren't welfare, people would be more motivated to work because there would not be that monthly income. Due to the nature of survival, if welfare weren't available, then a person would work." Three respondents interpreted the statement to imply that people on welfare are lazy.

Recommendation:

Retain statement wording.

Round 1:
Tested wording:

Welfare helps people get on their feet when facing difficult situations such as unemployment, a divorce, or a death in the family. Do you strongly agree, agree, you're in the middle, disagree or strongly disagree?

Problems:

The statement about welfare helping people get on their feet was confusing to some respondents as they seemed to focus on the examples and think of specific cases they knew about (e.g., they don't give you enough for a funeral or death). In addition, the word "help" was interpreted as the level of help; that welfare didn't help enough. Persons answering "in the middle" seemed to be thinking of welfare as not helping them get on their feet.

Recommendation:

Revise question wording and retest in Round 2. (We were unable to develop question wording with which we were satisfied so we retested the original wording again in Round 2.)

Round 2:
Tested wording:

Welfare helps people get on their feet when facing difficult situations such as unemployment, a divorce, or a death in the family. Do you strongly agree, agree, you're in the middle, disagree or strongly disagree?

Problems:

Most of the respondents referred to the list only as examples of hardships cases where a person might rely on welfare for a temporary time. Many respondents added that welfare is a short term solution. It seems for the people "in the middle" that the opposite of "getting on their feet" is when people are using/abusing the system as a way of life. Differences in results between Round 1 and Round 2 may be due to differences in the respondent pool.

Recommendation:

Retain statement wording.

Round 1:
Tested wording:

Welfare encourages young women to have babies before marriage. Do you strongly agree, agree, you're in the middle, disagree or strongly disagree?

Problems: