SPD Working Paper SPD97-4: Pretest Evaluation Report 1997

Survey of Program Dynamics

Pretest Evaluation Report

Prepared by: Jennifer Hess

Jennifer Rothgeb
Andy Zukerberg

Center for Survey Methods Research

19 December 1997

SURVEY OF PROGRAM DYNAMICS

Pretest Report
Executive Summary

In August 1996, welfare reform legislation was passed and the Census Bureau was mandated to conduct a survey to evaluate welfare reform and its impact on the nation. The Survey of Program Dynamics (SPD), a longitudinal demographic survey, is designed to accomplish that goal. The SPD uses a sample of original respondents from the 1992 and 1993 panels from the Survey of Income and Program Participation (SIPP). The SPD is conducted each Spring from 1997 through 2001.

The first implementation of the questionnaire designed specifically for the SPD is scheduled for Spring 1998. There are two components to the SPD. The "core" survey instrument obtains information on such topics as employment, earnings, income sources and amounts, program participation, eligibility, educational enrollment, job training, disability, health care utilization, health insurance, children's enrichment activities, child care, child support, contact with absent parents, food security, marital relationship and conflict, and adult depression. Most questions in the survey are about the past calendar year, with some exceptions. The second component of the SPD is a self-administered questionnaire (SAQ) designed for persons 12-17 years of age. The SAQ is designed for administration through a personal audio-cassette player. Topics covered in the SAQ include family routines, housework and chores, relationship with parents, parental monitoring, contact with nonresidential parent, minor problem behaviors, substance use, knowledge of and attitudes towards welfare programs, marriage and childbearing, sexual initiation and contraception. (Adolescents 12-13 years of age are not asked the sexual initiation and contraception questions.)

The primary purpose of the SPD pretest was to evaluate the SPD survey instrument, the adolescent SAQ, and some of the logistical, operational, and procedural aspects of the survey. Another purpose of the pretest was to obtain timer data so we could determine whether the length of the survey necessitated cuts in content prior to the 1998 SPD. Field pretesting of the SPD was conducted between October 6-22, 1997 in four regional office areas: Boston, Kansas City, Los Angeles, and Atlanta. Personal visit interviews were conducted using computer-assisted personal interviewing (CAPI). Pretest evaluation sources include debriefings of Field Representatives, interviewing observation reports, and review of taped interviews in which FR/respondent interactions were analyzed to detect problematic items. In addition, response analysis and respondent debriefing analysis was conducted (by Child Trends, Inc.) on the data obtained from the adolescent SAQ.

This report details recommendations for changes in question wording and sequencing. We also provide suggestions for enhancements to FR training and the FR manual. Some procedural issues, such as, the use of respondent flashcards and the labor force activity worksheet are also addressed.

The majority of suggested revisions are minor question wording changes. Nearly all of these recommendations were agreed to by the sponsoring divisions (POP/HHES) and can be implemented in time for the 1998 SPD. In some cases, our suggested revisions include more complex changes to the sequencing of questions within a series or between series. We also requested changing a series currently designed at the person level to a household level design, in an effort to reduce both FR and respondent burden. Many of these structural suggestions were agreed to by the sponsoring divisions, however, most of the more complex revisions cannot be implemented for the 1998 SPD due to resource and time constraints. They will be held for 1999 implementation.

Given the size of the SPD instrument and the quantity of suggested revisions, it is not possible to summarize all the recommendations in the executive summary. Among the more significant recommendations resulting from the pretest are the following:

Eliminating the Labor Force Activity Worksheet, replacing it with a calendar similiar to that used in SIPP;
Resequencing the labor force series, so data for last year are collected prior to data for "last week";
Providing a response option of "retired" in selected labor force items and skipping such persons out of inappropriate items to reduce interviewer and respondent burden;
Reordering the employment series so the name of employer is obtained prior to the "employer loop" (which obtains detail information for each employer);
Including household-level income screeners for the income source module to reduce the number of irrelevant questions asked of specific income groups;
Including a household-level income screener for high income households for the food security module to eliminate unnecessary respondent burden;
Screening out inappropriate subgroups (men, never divorced adults, etc.) from questions regarding WIC, alimony, disability income, etc;
Revising the item that identifies type of health insurance coverage to be asked at a person level instead of a household level to reduce interviewer and respondent burden;
Revising question wording in almost all modules to clarify concepts and terms to increase respondent comprehension and reduce confusion, to reduce task difficulty and decrease burden, to clarify reference periods, etc.

The report also details the successful administration of the adolescent supplement. Very few revisions are recommended for the 1998 SAQ.

Pretest timer obtained from DSD indicates that no cuts to the core SPD are needed to meet the targeted 60 minute per household interview time for the 1998 SPD. Timer data revealed the average core pretest interview took 55.5 minutes, once outliers on both ends were removed. (The average when outliers were not removed was 62.19 minutes.)

Table of Contents

I. Background

A. Purpose
B. Pretest methodologies and procedures II. Question evaluation methodologies A.Interviewing observation reports
B. Field Representative debriefings
C. FR and respondent interaction analysis III. Recommendations A. Core questionnaire

Employment
Income sources
Income amounts
Eligibility
Educational enrollment
Work training
Functional limitations and disability
Health care utilization
Health insurance
Food Security
Children's school enrollment
Children's enrichment activities
Children's disability
Children's health care utilization
Designated parent's work schedule
Child care
Child support
Contact with absent parents
Marital Relationship and Conflict

Adult Depression

B. Adolescent questionnaire

C. Respondent Flashcards

D. Timer Data

IV. Appendices

I. BACKGROUND

In August 1996, welfare reform legislation was passed and the Census Bureau was mandated to conduct a survey to evaluate welfare reform and its impact on the nation. The Survey of Program Dynamics (SPD) is an omnibus data collection vehicle that provides the basis for an overall evaluation of welfare reforms. The SPD is a longitudinal demographic survey designed to collect data on the economic, household, and social characteristics of the U.S. using a sample of original respondents from the 1992 and 1993 panels from the Survey of Income and Program Participation (SIPP). The SPD will be conducted each Spring from 1997 through 2001. (In Spring 1997, a modified CPS March Income supplement survey instrument was administered to the SPD sample to collect income and program participation data for calendar year 1996, thereby "bridging" the gap between years in which SIPP data were collected (1992 -1995) and SPD data will be collected (1997-2001.) The first implementation of the questionnaire designed specifically for the SPD is scheduled for Spring 1998.

Prior to conducting a field pretest, cognitive interviews, using the paper version of the questionnaire, were conducted for the following modules of the SPD survey instrument: educational enrollment, work training, functional limitations and disability, health care utilization, health insurance, children's enrichment activities, child care, child support, and contact with absent parent. Respondents with targeted characteristics, appropriate to the related modules, were recruited for one-on-one, in-depth think-aloud interviews. Researchers asked respondents probing questions to determine their understanding of specific questions and related concepts, item sensitivity, and task difficulty. Based on results of cognitive interviewing, recommendations for question revisions were proposed. Prior to the pretest, cognitive interviews were also conducted using the SAQ. Agreed upon revisions were incorporated into the SPD pretest instrument.

We were unable to conduct cognitive interviews for the "economic" modules of the SPD instrument (employment and earnings, income sources and amounts, and eligibility) since these modules are much too complex to administer using the paper version of the questionnaire. The automated instrument was not available within the timeframe needed for cognitive testing of these modules. Over half of the survey instrument is covered by these modules so it is not surprising that the pretest data revealed a disproportionate number of problems with these sections relative to the sections that had been cognitively tested and revised prior to the pretest.

Field pretesting of the SPD was conducted between October 6-22, 1997 in four regional office areas: Boston, Kansas City, Los Angeles, and Atlanta. Personal visit interviews were conducted using computer-assisted personal interviewing (CAPI). Eligible household respondents were household members 15 years of age and older. Census Bureau Field Representatives (FRs) completed 262 household interviews of the SPD core CAPI survey instrument and 60 adolescent self-administered questionnaires (SAQ). The pretest sample was selected from expired March 1996 Current Population Survey (CPS) interviewed households. Low income households were over sampled to increase the likelihood of covering more paths through the SPD questionnaire. Attempts were made to exclude non-English speaking households in the pretest sample, since only an English version of the survey instrument existed.

A. Purpose

B. Pretest methodology and procedures

The 1997 SPD pretest included questions on the following topics for household members age 15 and over: employment, earnings, income sources and amounts, program participation, eligibility, educational enrollment, job training, disability, health care utilization, health insurance (all household members), food security (all household members), marital relationship and conflict (respondent only), and adult depression (respondent only). The pretest questionnaire also included the following topics about children: school enrollment, enrichment activities, disability, health care utilization, child care, child support, and contact with absent parents. Most questions in the survey are about the past calendar year, with some exceptions. This part of the questionnaire was administered using CAPI and is referred to as the "core" questionnaire. (The paper version of the SPD core questionnaire, dated August 28, 1997 is the document to refer to when reading this report.)

The 1997 pretest also included an adolescent SAQ for persons age 12-17. The SAQ was administered through a personal audio-cassette recorder if the adolescent was at home when the FR administered the core questionnaire. If the adolescent was not at home, the FR collected the data by phone. Census Bureau staff developed two answer booklets: one for the adolescent to use with the audio cassette recorder, and a second for FRs to use during telephone administration. The former included the response categories but did not include the questions, since these were on the tape. This was done to protect adolescents' privacy. The latter included both the questions and the answer categories. Prior to administering the questionnaire, FRs obtained verbal consent from parents to allow the adolescent to participate in the survey. Parents who were interested were given a copy of the survey questions to review. To protect adolescent's privacy, the parent booklet contained the questions only and in a slightly different order than the adolescent answer booklet.

Reports from regional offices, Field Representatives, and staff who observed interviews were used to evaluate survey procedures, forms, manuals, and data collection instruments. Timers were included in the instrument to determine the interview length.

II. QUESTIONNAIRE EVALUATION METHODOLOGIES

We used three methods to evaluate the SPD pretest questionnaire: interviewing observation reports, debriefings, and interviewer and respondent interaction analysis (from taped interviews). Each method is described more fully below.

A. Interviewing observation reports

Eight Census Bureau staff observed pretest interviews (this includes staff from Demographic Surveys Division (DSD), Population Division (POP), Housing and Household Economic Statistics Division (HHES), Demographic Statistical Methods Division (DSMD), and the Center for Survey Methods Research (CSMR) in the Statistical Research Division. A total of 25 interviews were observed: 7 in Boston; 9 in Los Angeles; 7 in rural areas of Minnesota, Missouri, and Iowa; and 2 in Miami. Each observer was requested to complete an "Interviewing Observation Form". Observers were requested to note major problems observed with the instrument, question wording or sequencing, procedures, respondents willingness to be tape recorded (for use in interviewer and respondent interaction analysis, see below), and other aspects of the survey instrument and procedures. Appendix A contains the Summary Report of Interviewing Observations, prepared from the individual Interviewing Observation Forms.

B. Field Representative debriefing sessions

Census Bureau staff conducted five debriefing sessions with FRs who participated in the SPD pretest: one each in Boston, Kansas City, and Miami, and two in Los Angeles. CSMR and DSD staff facilitated the first half of the debriefing regarding questionnaire design issues, and Field Division staff facilitated the second half regarding field procedures, training, manuals, instrument layout, function keys, and other field-related issues. Nearly all FRs who conducted interviews during the SPD pretest participated in a debriefing session; a total of 38 FRs were at the debriefings. Appendix B contains the Field Representative Debriefing Summary Report for the CSMR/DSD portions of the debriefings. In a separate document, FLD provided a summary of their sections of the debriefings.

C. Interviewer and respondent interaction analysis

FRs were requested to tape record two completed core SPD interviews for use in subsequent interviewer and respondent interaction analysis. We received 61 tapes that contained either completed or partial interviews. Of those, CSMR staff analyzed interviews in 28 households, which included 46 household members ages 15 and over, and approximately 10 household members under 15. The analysis consisted of listening to the interview and noting any problems FRs or respondents had with question wording, sequencing, unclear concepts or terms, difficult to answer questions or series of questions, and other behaviors or indicators of how well a question is measuring the concept of interest.

The data from the interaction analysis are qualitative. They indicate the item number, what the problems with the question were, and, usually, verbatim transcripts of the FR/respondent exchanges for the problem areas. For example, if a respondent requested clarification of a concept or question, we recorded the question number and the verbatim question the respondent asked. If an FR made a major change to question wording, we recorded the verbatim question reading. Due to time and resource constraints, we coded only those items that indicated a potential problem existed based on the FR/respondent interaction. (Typically, when we "behavior code" taped interviews, we code EVERY question administration, even if there is no problem indicated. This allows for discussion of some "quantitative-type" results.) We decided to use the more qualitative approach so we would have more depth of understanding of what the problems were, not just that a problem existed. (Traditional behavior coding does not usually identify the specific source of the problems, which makes it less useful for developing solutions.) Because there was little time to develop recommendations for revisions, we decided this modified method would better serve our needs. (Obviously, due to the small number of cases reviewed and some of the more rare paths taken through the instrument, there were several items for which no FR/respondent interaction information was obtained.)

III. RECOMMENDATIONS

This report details recommendations for changes in question wording and sequencing. The format is such that typically, the original question wording is provided, along with the suggested revision. The justification for the revision citing information obtained from various evaluation sources is provided along with the final decision resulting from meetings between CSMR, POP, HHES, and DSD. Instrument problems identified during review of taped interviews are also noted in the report. We also provide suggestions for enhancements to FR training and the FR manual. Some procedural issues such as the use of respondent flashcards and the labor force activity worksheet are also addressed.

CSMR staff reviewed data from the three evaluation methods described above for each question in the core SPD survey. In some cases, we only had data from one source, such as the taped interviews. In other cases, we had data from all three sources. The source of the data is indicated in the justification that accompanies each suggested revision.

A. Core Questionnaire

1. Employment section

a. Items 3, 4, 5, 6, and 7 (labor force status "last week"

Original wording: LAST WEEK, did you do ANY work for pay?

Suggested revision: Delete "last week" series.

Justification: Shifting reference periods in the employment section from last week to the past calendar year is confusing, particularly for households with more than one person age 15+. Problems with the shift in reference period within the series were noted by some FRs, interviewing observation reports, and in the interviewer/respondent interactions. Respondents sometimes needed clarification on what reference period they were reporting. Deleting the current labor force status would allow the core SPD interview to focus from the very beginning on the reference period of "last year," providing a smoother flow for the interviewer and respondent at the outset of the interview.

Taped interviews and interview observation reports indicated that persons working for pay-in-kind and/or with atypical work situations had problems responding to item 3 and FRs had difficulty knowing how to classify such cases. The data indicate that the wording of the current work question is too narrow in scope and leads to misclassification of atypical work situations. This then affects classification of labor force activity for the previous calendar year. For example, if a person is working for pay-in-kind and is marked as a "no" in the current work question, it is probably more likely that the FR and the respondent will assume the answer to item 9 is also "no", even though item 9 does not specify work for pay. We suspect that with increased movement from welfare to work, programs will be established in which people may not be performing work for pay during the transition, but may be receiving other types of compensation.

Final decision: To address these problems, we recommend moving the "current work" series AFTER the entire employment series for last year has been collected for all persons age 15 or over in the household. This would allow the interview to focus on the previous calendar year, at the beginning of the core interview, which is the intended frame of reference for the survey. In addition to re-sequencing, we also recommend revising the wording of item 3, as follows:

"Did you do any work at all LAST WEEK, including work for pay or any other type of compensation?" This wording was included in a large-scale test during the CPS questionnaire redesign research. This question wording produced more "correct" responses to vignettes of marginal and atypical work situations posed to respondents, than the "work for pay" question contain in the pretest questionnaire. Results from other evaluation sources (behavior coding, response distribution, item nonresponse, FR debriefings) produced positive results regarding this wording.

Training Issues: FRs need to be provided with exercises on how to classify atypical work situations: e.g., persons working for pay-in-kind, persons working on commissions who did not receive any commissions during the reference period, etc. In these cases, respondents frequently "tell their story" and assume the FR knows how the situation should be classified. The taped interviews collected during the pretest for behavior coding could be a useful resource for developing training materials.

b. Items 4 (with a job, not at work), 5 (layoff), 10 (seasonal/part time work last year)

Original wording: Current response options include only "yes," "no," "don't know," and "refused."

Suggested revision: For persons 50+, include a response option of "retired" below the "no" option. For persons who voluntarily report being retired in item 4, skip them to item 9. For persons who voluntarily report being retired in item 5, skip them to item 9. For persons who voluntarily report being "retired" in item 9, ask item 10 to ensure they didn't do any part-time, temporary or seasonal work. If these same persons report "no or retired" in item 10, plug item 18 "retired" and go to CK64.

Justification: All three evaluation sources indicate that questions on employment are extremely burdensome for retired persons. Based on the research conducted during the redesign of the CPS, we recommend adding a "retired" response option (for persons 50 years of age or older) to items 3, 4, 5, and 10. This revision would reduce both interviewer and respondent burden.

Final decision: Suggested revision accepted. Note that HHES agreed to move items 3-7 to the end of the employment and earnings series and to ask these items after the calendar year information has been collected for everyone age 15 or over in the household. Therefore, answers of "retired" in items 4 or 5 would result in asking the current labor force status for the next eligible person in the household. Once this is complete, the instrument should go to income sources.

c. Labor force activity worksheet

We recommend that the labor force activity worksheet be eliminated from the SPD survey. There is strong evidence from FRs, observers, and taped interviews that the worksheet is not useful in the majority of situations. Based on FRs suggestions, we recommend that the worksheet be replaced with a calendar similar to that used in SIPP.

Final decision: Recommendation to eliminate labor force activity worksheet was accepted. It was decided to use the SIPP calendar with the numbered weeks.

d. Item 18 (main reason not working)

Original wording: What was the MAIN reason you did not work in 1996?

Suggested revision: Add a response option of "never worked" above the "other" response option. Add a flashcard for this item.

Justification: FRs reported that this item was problematic, particularly for persons who have not been in the labor force (specifically, elderly persons and adolescents who have never worked.) Other respondents are unclear what kind of answer we are looking for in response to item 18. To resolve these issues, we recommend adding a response option of "never worked." We also recommend adding a flashcard so respondents will have a frame of reference when answering the question.

Final decision: Accept suggested revisions; however, the use of a flashcard at this item is a low priority. Per HHES's request, revise the order of the response options in Item 18 so the more common reasons are at the top of the list, as indicated below:

<1> Retired

<2> Taking care of home or family

<3> Going to school

<4> Ill or disabled

<5> Could not find work/no work available

<6> Did not want to work

<7> On layoff

<8> Never worked

<9> Other (specify)

<D,R>

e. Items 19 and 32

Instrument problem: FRs indicated that these items need more pairs of answer spaces for workers with more than 7 different spells of employment/job search last year.

f. Item CK32

Instrument problem: The first condition of this check item (If 29 eq 1, go to 35) did not seem to work. Single jobholders were erroneously being asked item 32.

g. Item 32 (Weeks worked/main job)

The concept of main job is currently defined in SPD as it is in CPS (the job at which the person worked the most hours). The concept of main job is relevant in CPS because we collect industry and occupation data for only one job even if the person holds more than one job. In SPD, we ask about the weeks worked, usual hours worked, industry and occupation (I/O), and earnings for up to four employers during the previous calendar year. Therefore, the concept of main job is not relevant in the SPD. We recommend that FRs ask about the jobs in the order in which the person worked the most weeks, second most weeks, third most weeks, etc.

Final decision: Recommendation accepted

h. Item CK39

Instrument problem: The condition that "if 52 not marked in question 32 for this job, go to 39" needs to be revised. For single job holders, the instrument should look at information from both item 19 and item 21 to see if week 52 was entered in either of those items. (They might create out variables that sum the information from these items.) For multiple job holders, use information from item 32.

i. Items 41-43 (name of company/business) and E-review screen

Original wording: (41) What was the name of the (company/nonprofit agency/government agency) for which you worked?

Suggested decision: (29a) What is the name of the employer for which you worked (the most weeks) in (fill LAST YEAR)?

Justification: We suggest collecting the name of up to four employers immediately after item 29 (item 29a above). If this recommendation is adopted, items 41-43 will be deleted. This revision allows the employer's name to be included as a fill in the loop of questions asked about each job for which the respondent is reporting. This includes hours worked, weeks worked, class of worker, and industry and occupation for each job. This revision was considered prior to the pretest, and the need for the revision was confirmed by the comments obtained from FRs during debriefings.

It should be noted that the current design allows the company name questions (items 41-43) to be filled based on responses to the class of worker question (item 35). The suggested revision requires that a generic company name question be used, since class of worker data will not be collected until later. However, the benefit of having the company name fill in all the questions within the employer loop series seems to offset the issue of not having the company name questions tailored according to class of worker status.

This revision will allow the E-review screen to list employers' names as opposed to "employer 1", "employer 2", etc. which is the current design. (This is the screen that allows FRs to delete an employer that was entered erroneously.) During the debriefings, FRs commented that the E-review screen, in its current form, is useless.

Final Decision: Modify item 29a to read: "What is the name of the employer or company for which you worked (the most weeks, second most weeks, etc.) in (fill LAST YEAR)?"

Allow a precode (with a specifiy) at Item 29a to indicate that "self employed, no company name" exists. For subsequent questions where name of employer is used as a fill, use "at this job" (in items 35, 46, 47 and 50) and "for yourself" (in items 32, 33, and 38), as appropriate. (You may need to look more closely at the instrument to determine if certain items will need additional fill instructions due to a lack of employer name.)

Training Issues: FRs need more and better training on how name of employer should be filled for persons working for various employers such as substitute teachers, domestic workers, odd-job handypersons, persons that are self employed, etc. HHES also needs to provide DSD/FLD with consistent criteria that can be applied (for item 29) to make a determination whether persons in such situations are multiple jobholders or single jobholders.

j. Item 43A (address of employer)

Original wording: What was the address?

(Currently, the instrument requires street address, city, state and zip code)

Suggested revision: In what city (town) and state is this employer located?

(Ask AFTER industry type (item 44) is obtained)

Justification: Information from the three evaluation sources strongly indicate that respondents are unable to provide the exact address of employers, including street address and zip code. As suggested by FRs, we recommend obtaining only the city (town) and state of employers. We recommend that this item be placed immediately AFTER item 44 and labeled item 44a.

Final Decision: Use original wording and if a "don't know" response is provided, then follow up with a probe for the city and state, such as:
"In what city(town) and state is this employer located?"

Training Issues: FRs need additional training on the purpose of this item.

k. Item 45

Instrument problem: The last fill of this item should reflect the last month in the previous calendar year that the person worked. Instead, the first month was appearing. (e.g. for persons working all year the fill should be December 1996; instead it was January 1996.)

l. Item CK49

Instrument problem: This check item might need to be revised. If the instrument goes through the employment/earnings series for more than one person in the household, this check item is suppose to skip over the lead-in and the request to refer to records which is contained in item 49. However, this item was appearing when there was more than one loop through the series.

m. Item 59 (pensions)

Original wording: Did this employer offer a pension or other type of retirement plan to any of its employees during 1996?

Suggested revision: During 1997, did this employer offer a pension or other type of retirement plan to ANY of its employees?

Justification: Respondents frequently responded "no" to the question, but it is unclear if they thought the question was asking about ANY of its employees or only about benefits offered to the sample person. Sometimes FRs were not reading the phrase "to any of its employees during 1996." Additionally, it is unclear if respondents always understood that the reference period was last year. For clarification, we recommend revising the question wording so the reference period is at the beginning of the question and the word "any" is emphasized.

Final Decision: Suggested wording accepted.

n. Item 60 (pension participation)

Original wording: Did you participate in that plan?

Suggested revision: During 1997 did you participate in that plan?

Justification: For clarification purposes, we recommend that the reference period be included in the question.

Final Decision: Suggested wording accepted.

o. Item 61 (health insurance)

Original wording: Were you eligible for health insurance coverage through this employer?

Suggested revision: During 1997 were you eligible for health insurance coverage through this employer?

Justification: For clarification purposes, we recommend that the reference period be included in the question.

Final Decision: Suggested wording accepted.

p. Item 62 (health insurance participation)

Original wording: Did you participate in that plan?

Suggested revision: During 1997 did you participate in that plan?

Justification: For clarification purposes, we recommend that the reference period be included in the question.

Final Decision: Suggested wording accepted.

q. Farm-related income

Problem identified: FRs participating in the pretest in the rural sample areas in Minnesota, Missouri, and Iowa indicated that they didn't think farm income was adequately captured in the SPD. They suggested we develop a flashcard with farm-related income to improve reporting. Two different approaches could be taken. First, a separate question could be included at the end of the earnings series, just before item 59 (fringe benefits items). The March CPS includes the following item: "Other than farm income we have already talked about, did you receive any income from agricultural work done for others, recreational services, or government farm programs other than loans?" The question is given as an example only. CSMR does not like the wording of this question, but we are not familiar enough with the different types of farm income to suggest our own question. If such a question is included, we would need to develop additional follow-up questions to capture the income received. A second approach is that a flashcard (with information listing different types of farm income) could be developed for use in the earnings items (51 and 53). FRs could be instructed to use the flashcard when item 35 (class of worker) equals "working in a family business or farm" and information obtained in industry/occupation indicates the person works in farming.

Final Decision: Do not add a separate question. Do not add a flashcard. The population to which this problem applies will be EXTREMELY small in the SPD sample.

r. Use of previously reported data (dependent interviewing)

FRs and other Bureau staff have suggested including dependent interviewing for employment information to improve the efficiency of the instrument. Since the input for the 1998 SPD is the Bridge Survey, which is the Current Population Survey and its March Income Supplement, there is limited employment information to include in dependent interviewing. Additionally, any specifications prepared for the 1998 SPD instrument would have to be re-written for 1999 since the 1998 SPD will be used as input for the 1999 instrument.

The Bridge Survey included basic CPS, which asked about labor force activity "last week," (i.e. the week before the interview). Industry and occupation data is collected for the "main" job, that is, the job at which the person worked the most hours last week. We could use those data for dependent interviewing. However, we need to consider whether the effort to include dependent interviewing is an effective use of resources considering the limited amount of time available to modify the pretest instrument for 1998 SPD.

Final Decision: Do not use dependent interviewing in 1998 for employment information. For 1999, explore use of previously recorded data from the employment series.

s. Other Items in the Employment and Earnings Series - Training Issues

Item 50 (Earnings last year).

Review of taped interviews indicated that respondents who only worked part year at a job sometimes report the "annual salary" instead of "annual earnings," even though they did not work the whole year. This results in an overestimate of earnings received. FRs need additional training for such cases so they know to probe to determine if the earnings being reported are earnings the person actually received or if what was reported was the earnings had the person stayed the entire year at the job.

For earnings items, training needs to be enhanced to emphasize to FRs that earnings BEFORE taxes is what's being requested. Also, FRs might need to be reminded of the definitions of Gross and Net earnings.

2. Income sources

FRs strongly requested use of income screeners to reduce the liklihood of inappropriate questions being asked of high income households (e.g. program participation) and low income households (e.g. income from shares of stocks, royalties, trusts, etc.) . The need for these screeners was echoed in the interviewing observation reports, as well as the taped interviews. Using household level income screeners (with criteria determined by HHES) will reduce both FR and respondent burden. Such a revision will most likely reduce requests for clarification for terms unfamiliar to respondents, thereby improving the interview flow.

a. Household level income screener

We suggest that low income households (to be defined by HHES) be screened out from individual questions regarding the following income sources:

Royalties (item 254)
Estates or trusts (item 256)
Mutual fund shares or shares of stock (item 249)
Properties that were rented to others such as houses, apartments, business properties, or land (item 251)

We recommend that high income households (to be defined by HHES) be screened out from individual questions for the following income/program participation sources:

Supplemental Security Income (items 211 and 214)
AFDC (item 220)
WIC (item 220)
General Assistance (item 220)
Other welfare (Item 220
Free or reduced-price meals at school (item 223)
Any financial assistance on a regular basis from friends or relatives not living here (item 262)
Income assistance from a charitable group (item 264)
Food stamps (item 218)
Energy assistance (item 228)

Final decision: Accept recommendation to adopt household level income screener. Specifications for the income screener will be provided to DSD/CSMR by HHES. Most likely the screener will be from the administration of the "family income" question, developed for use in SIPP.

b. Catchall Questions

It is necessary to administer two "catchall" questions at the end of the income sources series: one for low income households, and one for high income households. Provided below are suggestions for such questions. Low-income households Suggested wording: Item 270: FLASHCARD X. This is a list of income sources persons sometimes have. Please tell me if anyone in this household received income from any of these sources during 1997.

FR: Read if necessary: Did anyone own mutual funds or shares of stock, own property that was rented to others, receive rental income from boarders, receive income from estates or trusts, or from royalties?

(If yes, ask:)

Item 271: What was the source of income?

(Mark all that apply)

a. Mutual funds or shares of stock..................0
b Property that was rented to others...............0
c. Rental income from boarders......................0
d. Income from estates or trusts......................0
e. Income from royalties.................................0 Final Decision: Adopt suggested wording, however remove response option "Rental income from boarders" out of the catchall question for low-income households. Inquiries about receiving income from this source will be asked in all households regardless of income level.

High income households

Suggested revision: Item 272. FLASHCARD X.

This is a list of benefits or income sources people sometimes receive. Please tell me if anyone in this household received benefits during 1997 from any of these sources. (If yes, ask:)

Item 273. What was the source of the benefits?

(Mark all that apply)

a. Supplemental Security Income (SSI)
b. AFDC, ADC, or TANF
c. WIC
d. General assistance
e. Free or reduced price lunches at school through the Federal School Lunch or Breakfast Programs
f. Food Stamps
g. Energy assistance
h. Income assistance from a charitable group
i. Other welfare
j. Financial assistance on a regular basis from friends or relatives not living here Final Decision: Suggested wording adopted. In response option "j", bold the words "on a regular basis" in the instrument and on the flashcard.
c. Item level screeners for inappropriate items. For a few items in the income source series, the questions are inappropriate for certain subgroups. Such items are identified below.

Item 258 (Receipt of alimony): This question should not be asked if all household members 15+ reported earlier (item in the demographics) that they have never been divorced.

Item 220 (WIC): This question should not be asked if the household contains no children under age 6 and no women of childbearing age (15-45).

Item 240 (Disability income): This question should not be asked if it was reported in item 18 that the sample person never worked.

Final Decision: Suggestion to screen out inappropriate subgroups from these items is accepted.

d. Item 228

Original wording: Has this household received any energy assistance in the past 12 months, that is since (month, year)?

Suggested revision: The government has an energy assistance program that helps pay heating costs. This assistance can be received directly by the household or paid directly to the electric company, gas company, or fuel dealer. During the past 12 months, has this household received any energy assistance of this type?

Justification: Information from the interviewing observation reports and from the taped interviews demonstrate that several respondents had difficulty understanding what was meant by the term "energy assistance." To improve respondent understanding, we recommend that the lead-in explanation used in the March CPS Income Supplement be added to the SPD question and the question wording revised slightly.

Final decision: Adopt suggested revision, but delete the second sentence to make the question less wordy, as shown below.
"The government has an energy assistance program that helps pay heating costs. During the past 12 months, has this household received any energy assistance of this type?"

In an FR note or a help screen add the following information:
"This assistance can be received directly by the household or paid directly to the electric company, gas company, or fuel dealer."

Training Issues: Revise FR training to include the definition of energy assistance and what should and should not be included as energy assistance. This information is available from HHES.

e. Items 236a and 236c Original wording: Item 236a (Do you/does anyone in this household) have a physical, mental, or other health condition that prevents (you/him/her) from working?

Item 236c (Do you/does anyone in this household) have a physical, mental, or other health condition that limits the kinds or amounts of work (name/you) can do?

Suggested revision: Delete item 236c

Problem identified: Since question 236c is broader than 236a, we recommend using this single question to capture the information currently obtained through 236a and 236c, thereby deleting question 236c from the survey. While 236a asks whether there is a condition that prevents (name) from working, we think situations prompting reports of yes to item 236a will produce reports of yes to item 236c. We have no evidence on which to base this opinion. But we do know that the current series seemed extremely redundant and unnecessary.

Final decision: Retain both items.

f. Item 236C Instrument problem: The first fill for this item (Other than you/names,) did not seem to work correctly. It seemed that if the person's line number was entered in 236B, the lead-in fill was left blank and consequently the same person reported in 236a would be reported in 236c. g. Item 238 Original wording: Did (you/anyone in this household) ever retire or leave a job for health reasons?

Suggested revision: Did (you/anyone in this household) ever retire or permanently leave a job for health reasons?

Review of taped interviews indicated that some respondents reported "yes" to this question when they temporarily left a job due to health reasons such as maternity leave or hospitalization. We assume the intent is to capture whether persons have permanently left a job for health reasons. If our assumption is correct, we recommend including the word "permanently" in the question so it is clear to respondents what is being asked.

Final decision: Suggested revision accepted.

h. Item 240 Instrument problem: During review of taped interviews it was observed several times that this item did not always appear when it should have. Sometimes persons for whom a yes was entered in 238 were never asked this question i. New item for types of assistance not covered elsewhere Research conducted by CSMR (Bogen, Lee, and Griffiths 1997) on the CPS March Income Supplement indicates that there are some types of assistance not detected through the usual battery of program participant/income sources questions, particularly transportation assistance and child care assistance. Based on the results of the March Income Supplement research, we recommend adding a new item to the core SPD questionnaire immediately after item 220. CSMR cognitively tested these items for the March CPS Supplement.

New item: At any time during 1997, did anyone in this household receive any of the following types of GOVERNMENT assistance because your income was low:

Transportation assistance, such as gas vouchers, bus passes, or help registering, repairing, or insuring a car?
Any child care services or assistance last year so you could go to work or school or training?
Any other assistance from the government last year, that we have not already talked about, because your income was too low? (If yes, specify)

For each type of assistance reported above, an additional question is needed to determine who received such assistance.

If the recommendation for including this new item is adopted, we recommend deleting the question concerning "other welfare" from item 220. Related items pertaining to who received the assistance will need to be modified. Additionally, this question would not be asked of high-income households if the income screen is adopted, as previously suggested.

Final decision: Do not delete "other welfare" from item 220.

Arthur Jones (HHES) provided specifications to DSD regarding additional types of assistance including assistance with finding a job. These specs include information regarding the above types of assistance, who received it, and quantity or frequency of assistance received. j. Item 244 (response category modifications) Original wording: <6> Regular payments from annuities or paid up insurance policies

<7> Regular payments from IRA, KEOGH, or 401(k)

Suggested wording: <6> Regular income from annuities or paid up insurance policies

<7> Regular income from IRA, KEOGH, or 401(k)

Justification: HHES requested this change to clarify retirement income.

Final decision: Suggested wording accepted.

k. Other Income Source Items - Training Issues Item 246

Enhance training to address savings bonds people may have.

Item 249

Enhance training so it is clear to FRs what should and should not be included for this item. Specifically, instruct them on how to handle retirement funds. The rewording of the question should help, but various scenarios will probably still be presented to FRs and they will need to know how to classify the response.

Item 260

Some FRs confused child support with foster child care. Enhance training and scenarios to include such situations so distinction is clear to FRs.

Item 261

Enhance training so FRs are aware that 261 (who receives child support) is about receipt of support by a parent, not the child for whom receipt is provided.

Items 206 and 212

Respondents are OFTEN confused as to whether they receive Social Security benefits or Supplemental Security Income (SSI) benefits. Review of the taped interviews indicates that respondents frequently give the FR information expecting the FR to know how to classify the response. Enhance training so FRs are more familiar with the differences between Social Security and SSI.

Income amounts

a. Item 300 (Verification of income sources)

Original wording: I have recorded that you received (READ LIST)

[LIST INCOME SOURCES IDENTIFIED FOR THIS PERSON}

Have I listed anything that should not be there?

<1> Yes

<2> No

Suggested revision: I have recorded that you received (READ LIST)

[LIST INCOME SOURCES IDENTIFIED FOR THIS PERSON]

Is that correct?

<1> Yes

<2> No

Justification: This item was counter to FR expectations, which resulted in erroneous entries. With the original wording, if everything was correct, the FR needed to enter a <2> for no (that nothing needed to be changed.) However, FRs were rewording the question to verify that everything was correct and then entering a <1>, which, with the current wording, indicated that something was listed that needed to be changed. FRs and respondents were confused when item 301 appeared asking for what should be deleted. FRs had to back up and change the response to item 300. Due to this problem, we recommend that item 300 be reworded.

Final decision: Recommendation accepted.

b. Item 302 (Lead-in to income amounts)

Original wording: Now I am going to ask you how much you received from each of these sources during 1996 and which months you received it. The worksheet you filled out regarding your work-related activities may be helpful in answering these questions as well as any records you have.

Suggested decision: [Fill "this" if only one income source is listed. Otherwise fill "these."]

Now I am going to ask you how much you received from (each of these/this) source(s) during 1997.

Justification: The Labor Force Activity Worksheet is not recommended for use in 1998, therefore, all references to the worksheet should be deleted from the instrument. Taped interviews indicated that the lead-in was awkward in cases where persons only had one income source, since it referred to multiple income sources. We recommend deleting the reference to which months the income was received since the months a particular type of income was received is only asked for selected income sources.

Final decision: Recommendation accepted.

c. Item 302

Instrument problem: This lead-in is only suppose to appear the first time through the amounts series. However, sometimes it appeared for the second and third persons through the series.

d. Item 445 and 449

Instrument problem: The text for the fill regarding type of interest-earning accounts being asked about should be based on the responses to item 246. If 246a is yes, then the fill should read "Savings, or interest-earning checking account, or money market fund. If 246b is yes, then the fill should read "bonds, treasurey notes, certificates of deposit). If both 246a and 246b are yes, then both sets of text should be included in the fill.

e. Item 449 and 450 (Amount of interest earned; Average amount in account)

Original wording: (449) How much did you receive IN INTEREST from these sources during 1996, including even small amounts credited to accounts?

(450) What is your best estimate of the AVERAGE AMOUNT that you had in these accounts during 1996?

Suggested revision: (450) What is your best estimate of the AVERAGE AMOUNT that you had in these accounts during 1997?

(449) How much did you receive IN INTEREST from these sources during 1997, including even small amounts credited to the accounts?

Justification: From the taped interviews, it was evident that respondents were frequently confused when trying to respond to item 450, particularly when trying to estimate an average amount in checking accounts. Some respondents reported the amount of savings, instead of interest, when asked item 449. To reduce erroneous reports of interest earned, we suggest reversing the items, so that respondents first report the average amount in the accounts and then report the interest. These same comments apply to items 445/446, 458/459. Item 450 presents a very difficult task for respondents.

Final decision: Suggested wording adopted with a minor modification. In item 450 ask for the AVERAGE BALANCE, instead of the AVERAGE AMOUNT, as provided below:

(450) What is your best estimate of the AVERAGE BALANCE that you had in these accounts during 1997?

(Note: The reordering revision and wording revision also apply to items 445/446 and 458/459.)

f. Items 454,455, 456, 457, 458, and 459

Problem identified: The text for these items should be revised to reflect whether the person owns mutual funds or shares of stock, depending on the response to items 249a and 249b.

g. Other Income Amounts Items - Training Issues

Item 315

Sometimes an adult receives payments on behalf of a child. Other times the child receives benefits directly. Provide FRs with information about the different types of arrangements so FRs can determine how to classify the response.

Item 449

Include in training more information about how to fill item 449. Respondents who have savings bonds for their children don't always know the interest that has accumulated to date and they report the face value of the bond instead of the interest.

Item 484

Review of taped interviews revealed that child support income was sometimes reported as the child's income rather than the parent's income. Enhance training so FRs recognize when this happens and appropriately probe to obtain the correct information.

Item 506

Provide FRs with information of types of income that are considered "lump sum' payments and those considered "capital gains.".

4. Eligibility/Assets Questions

a. Item 604

Original wording: About how much would this (house/apartment) sell for if you were to put it on the market today?

Suggested revision: How much do you estimate this (house/apartment) would sell for if you were to put it on the market today?

Justification: Review of taped interviews indicate that many respondents had difficulty providing an answer to this question. Respondents often said they didn't know, then said they could only guess or provide an estimate. In the original wording, we attempted to communicate to respondents that an estimate was acceptable by including the phrase "about how much." Given the difficulty encountered by respondents and confirmed by the FRs in the debriefing report, we recommend revising the question to ask for an estimate.

Final decision: Use suggested wording.

b. Items 605, 606, 607, 608, 609, 617, and 618a

Original wording: (605) How much were your total property taxes, including city, county, and school taxes on this (house/apartment) in 1996?

(606) How much did you pay for homeowner's insurance, that is, what was your premium in 1996?

(607) Do you have a mortgage or home equity loan on this property?

(608) Including any second mortgage or home equity loan, about how much is the remaining principal on this mortgage?

(609) How much are your monthly mortgage payments?

(617) Do your mortgage payments include property taxes?

(618a) Do your mortgage payments include insurance premiums?

Suggested revision: (607) Do you have a mortgage or home equity loan on this property?

Lead-in: The next few questions are about your property taxes, homeowners insurance and current mortgage payments on this home. It will be much easier to provide this information if you refer to your mortgage statement or mortgage payment coupons. I'd be glad to wait while you get those records.

(609) How much are your monthly mortgage payments?

(617) Do your mortgage payments include property taxes?

(605) How much are your total property taxes, including city, county, and school taxes?

(618a) Does your mortgage payment include insurance premiums?

(606) How much do you pay for homeowner's insurance, that is, what is your annual premium?

(608) Including any second mortgage or home equity loan, about how much is the remaining principal owed on your current mortgage?

Problems identified: From the taped interviews, there was strong evidence that many respondents were unable to respond to these questions; the primary reason being that the taxes and insurance payments are included as part of the monthly mortgage payment. There was evidence that many respondents do not know the remaining principal on their mortgage and some respondents don't understand the term "principal." Unless respondents have their mortgage statements, payment coupons, etc. readily available, there will be high nonresponse for several of these items.

To facilitate use of records when reporting these data, we recommend reordering of the series and including a lead-in asking respondents to refer to their mortgage records. We suggest that the reference period of the question on how much is paid in taxes and in homeowners insurance be changed to current year instead of last year. We suspect that taxes and homeowners insurance don't change that much from year to year, so the effect on the data will be minimal. FRs indicated that there was some confusion with regard to item 608. Some respondents who have refinanced their homes several times thought the phrase "including any second mortgage" referred to the second one in a series of refinanced mortgages. With the frequency with which homes are refinanced due to shifting interest rates, we recommend revising question wording to eliminate potential respondent confusion.

Another issue regarding these items was raised at the FR debriefing in Minnesota. The FRs reported that many farmers don't know the property tax paid on their home because their tax bill includes both the house and the land. Often the land is more valuable. Also, some farmers don't know the amount paid for homeowners insurance because they have umbrella policies that include their home, cars, and other liabilities. Neither of these problems are unique to farmers, other than their property may be much larger than non-farm houses. To minimize this problem, we recommend revising the property tax question to delete the direct reference to "this house."

Final decision: Modify question wording, sequencing and delete some questions.

HHES determined that some of the original questions contained in this series were not needed to determine program eligibility, but are instead used to determine wealth/net worth. Since the wealth/net worth information is not critical for the purposes of SPD, some of the items can be deleted including: the question pertaining to remaining principal on the mortgage and the separate questions asking for the amount of property taxes and homeowners insurance. (For persons whose mortgage payments don't include taxes and insurance, separate questions on property taxes and homeowners insurance will continue to be asked.)

The agreed upon sequencing and wording of this series is provided below.

Do you have a mortgage on this property?

<1> Yes

<2> No

Do you have a home equity loan on this property?

<1> Yes

<2> No

CK609. If 607= "1", ask 609.

Otherwise go to 611.

How much are your monthly mortgage payments (including any condo or association fees)?

________.00

Do your mortgage payments include property taxes?

<1> Yes (GO TO 612)

<2> No

611. How much are your total property taxes, including city, county, and school taxes?

________.00

CK612. If 607= "1", ask 612.

Otherwise go to 613.

Does your mortgage payment include insurance premiums?

<1> Yes (GO TO CK614)

<2> No

How much do you pay for homeowner's insurance, that is, what is your annual premium?

________.00

CK614. If 608= "1", ask 614.

Otherwise to 618C.

What is the balance remaining on your home equity loan?

Training Issues: Item 605 (Property Taxes)

Train FRs that taxes on the housing unit and associated property is the information being requested. In some areas with farmland, respondents were unsure if they should include the acreage associated with their home.

c. Item 618C (utilities)

Original wording: How much did this household pay for electricity, gas, and other utilities last month?

Suggested revision: The next few questions are about amounts paid LAST MONTH for utilities such as electricity, water, and telephone service.

Last month, how much did this household pay for electricity?

And how much for gas or other types of heating fuel?

Last month, how much did this household pay for BASIC telephone service?

And how much for water and sewer and other utilities?

Justification: There was evidence from all three evaluation sources that the question asking for amount of utilities was problematic and time consuming. Respondents tend to report amounts paid for each utility separately. FRs had to sum the individual amounts (without a calculator) to obtain a total. Respondents also tended to report an average amount paid, rather than the amount paid last month. To reduce the interviewer burden for this item, we recommend splitting the item into separate questions for each relevant utility service and having an internal calculation of the total amount. Also, we suggest a lead-in be added to emphasize that we are asking about the amount paid last month.

Final decision: Ask for the "usual monthly" amount instead of the amount "last month." The revised wording is provided below.

The next few questions are about your usual monthly utility bills.

How much (do you/does this household) usually pay for electricity per month?

_________.00

How much for gas or other types of heating fuel per month?

_________.00

How much (do you/does this household) pay for BASIC telephone service per month?

_________.00

And how much (do you/does your household) usually pay for water and sewer per month?

_________.00

Training Issues: Emphasize to FRs in training that long distance charges are NOT to be included in the amount of BASIC telephone service.

Train FRs that the time frame covered by bills for water and sewer vary widely by area. Some bills are for a monthly period, but many areas bill every 3 months, 6 months, etc. FRs should be aware of this and trained to probe, when appropriate, to make sure respondents are reporting a monthly amount. This same issue applies to the question about "other types of fuel." It may be that a person only gets fuel once or twice during the winter. The FR and respondent will need to work together to estimate a monthly amount.

d (1) Item 631 (see item 635 for problem with this item)

Original wording: Does anyone in this household own a car, van, or truck, excluding recreational vehicles and motorcycles?

Final decision: Does anyone in this household own a car, van, or truck? Do not include leased vehicles, recreational vehicles, or motorcycles?

d(2) Item 635

Original wording: Is this vehicle owned free and clear or is there still money owed on it?

Problem identified: In the Interviewing Observation Report Summary, an observer noted that this question was difficult to answer for persons leasing vehicles.

Final decision: Use original wording. (Change made to item 631 to accommodate issue related to leased vehicles. HHES does not want leased vehicles to be counted as "owned".)

Training issue: Train FRs that leased cars are not considered "owned" and should be marked as "no" in item 631.

e. Item 637

Original wording: Is this vehicle used primarily for either business purposes or for the transportation of a disabled person?

Suggested decision: Not counting routine use to get to and from work, is this vehicle used primarily for either business purposes or for the transportation of a disabled person.

Justification: Information obtained during the FR debriefings and from the taped interviews strongly indicates that many respondents misinterpreted this question. Respondents were including routine use of their vehicle to go to and from work as using it for "business purposes." To remedy this misunderstanding, we recommend revising the question to clarify that routine use is not to be counted as business purposes.

Final decision: Use suggested wording.

f. Item 645

Instrument problem: The first fill for this item did not seem to work right. The fill is suppose to say "Aside from mortgages/Aside from car loans, etc." However, review of taped interviews demonstrate that frequently when the person had just reported still owing money on a car (in item 635), the fill did not appear. Also, when the person did not report having a car loan, the fill DID appear.

5. Educational enrollment

a. Item 701

Original Wording: At any time since September 1996 (were you/was name) enrolled in school, either full or part time?

Suggested revision: At any time since September 1997, were you enrolled in a school such as high school, college or vocational school?

Justification: The suggested revision is designed to focus respondents on enrollment in regular schools and reduce the likelihood of respondents answering 'yes' to school enrollment when they were in a job training program or other skill enrichment program not associated with regular school. The suggested wording is based on information gained through SPD cognitive interviews and was agreed upon in a meeting between POP and CSMR. However, this was not the wording used in the pretest instrument.

Additionally, information obtained from the FR debriefing indicates that some respondents reported they were in school when enrolled in English as a Second Language (ESL) class. Revising the question to include examples of the types of schools may reduce this problem. With increased transitions from welfare to work, it is likely that the number of persons enrolled in programs such as ESL will grow. If ESL classes are located at a local college or high school, it is not unreasonable that the respondent may report that he/she is enrolled in a high school or college.

Final decision: At any time between September 1997 and April 1998 (were you/was name) enrolled in school, either full or part time?

Training issues: Include better description in training and the manual regarding the types of educational activities that are counted as "school." For analytic purposes we want to learn about people taking credited classes that could lead to a degree. We are interested in knowing about these credited classes even if the respondent does not intend to complete the course work necessary for a degree. Frs need clarification of the difference between work training and vocational school.

b. Item 703

Original wording: What was the highest level at which you were enrolled?

Suggested decision: Was it a high school, college, vocational school or something else?

High school (Ask A)

College (Ask B)

Vocational, technical, business school beyond high school level

Something else (specify) (Go to Work Training)

A. In what grade are you enrolled?

High school grade 9-10

High school grade 11

High school grade 12

High school equivalency/GED program

B. At what level were you enrolled?

College year 1 (Freshman)

College year 2 (Sophomore)

College year 3 (Junior)

College year 4 (Senior)

College year 5 (first year graduate or professional school)

College year 6 (second year or higher graduate or professional school)

Enrolled in college, but not working towards degree

Justification: FRs indicated during the debriefing that some respondents had difficulty with the concept of "highest level at which you were enrolled." This was confirmed by confusion ("What do you mean highest level?"), long pauses, inaccurate responses, and requests for clarification identified in the taped interviews. This is consistent with the findings from the earlier cognitive interviews in which respondents had difficulty with the wording "highest level or grade at which you were enrolled."

To eliminate the confusion associated with obtaining highest level of school, we recommend asking a direct question with the types/broad levels of schools (high school, college, vocational school, or something else) included as the response categories. Then, the specific level of high school and college can be obtained with the same level of detail as the response categories in the original question.

Final decision: Use suggested wording. A statement needs to be included on a help screen (or item screen) that if a person switched levels during the reference period, the higher of the two levels should be marked in this item.

Training issues: A person taking classes to improve his/her work skills will be identified in the work training section and should not be included here. If a respondent reports work training in this section, it should be coded as >something else' and the instrument will jump into the work training section. However, choosing something else, will prevent FRs from asking the school enrollment questions. We also recommend that FRs be trained to mark persons enrolled in ESL classes as "something else" in the suggested revision for item 703. FRs should receive some training on how to determine level for college students. For example, does a respondent need a certain number of credits to be considered college year two?

c. Other educational enrollment training issues

Item 704

It is not clear from the current question how a respondent who indicates he/she receives assistance from other relatives (grandparents, uncle, aunt, sibling) should be coded in this question.

Work training

a. Item 805

Original wording: Where did you receive this (most recent) training?

Business, technical, or vocational school

High school

Two-year or community college

Four-year college or university

At current or previous employer's place of work

Correspondence course

Sheltered workshop

Vocational rehabilitation center

Other

Suggested revision: Same as above, but add "unemployment office" to list of response options.

Justification: In one taped interview, the respondent indicated receiving training from an unemployment office. If HHES thinks it will be useful analytically to capture this information, we recommend including a specific response category; otherwise, such cases can be recorded in the "other" category.

Final decision: Add "unemployment office" to list of response options.

b. Resequencing work training/educational enrollment sections

Problem identified: FRs indicated that some respondents reported work training activities earlier in the educational enrollment section. The error in reporting was usually uncovered within the work training series of questions. FRs then had to back up in the instrument to the beginning of the school enrollment questions and change all of the answers.

Suggested revision: We propose re-sequencing the instrument so the work training questions (Items 801-810) come prior to the educational enrollment questions (Items 700-705).

Final decision: DSD indicated this request is a low priority since TMO does not want to implement this unless they are certain it can be done without jeopardizing the total functionality of the instrument.

7. Functional Limitation and Disability

Problem identified: Information from all three evaluation sources indicated that the current design (person level) of this series is extremely burdensome for FRs and respondents. Taped interviews indicated that respondents frequently interrupted the FRs with answers of "no" before the question reading was finished. Additionally, FRs frequently paraphrased questions, sometimes leaving off key words or phrases, to avoid irritating respondents with multiple administrations of the series. Both FRs and interviewing observers suggested asking this series at the household level since respondents tend to >tune out' after they go through the series one time.

We suggest asking one series of questions for persons 15+ and one series of questions for children ages 0 to 14. The impact on data quality should be moderate. Reducing repetition may do a better job of holding respondent and interviewer attention than repeating the same series over and over again. This could have positive implications for the data collected later in the survey as respondents are less likely to be tired or bored. The suggested series for persons ages 15 and over is provided below. If our suggestion is accepted, item 900 regarding each person's health in general could be moved to the beginning of the health care utilization section, which is still asked at the person level.

Suggested revision:

For persons age 15-21, ask:

901. Because of a physical, learning, or mental health condition, (do you/does (fill names of persons 15 to 21 years old)) currently have any limitation in (your/his/her/their) ability to do regular school work?

<1> Yes ==> Ask "who" if more than one person 15+

<2> No

For persons age 15-21, ask:

902. (Did you/Did (fill names of persons 15 to 21 years old)) receive any special education services during the past 12 months, that is since (month, year)?

<1> Yes ==> Ask "who" if more than one person 15+

<2> No

For all persons age 15+, ask:

904. (Do you/Does anyone in this household 15 years of age or older) have difficulty seeing the words and letters in ordinary newspaper print even when wearing glasses or contact lenses?

<1> Yes ==> Ask "who" if more than one person 15+, and ask 905 for each person identified

<2> No (GO TO 906)

For each person identified from 904, ask:

(Are/ Is) (you / name) able to see the words and letters in ordinary newsprint at all?

<1> Yes

<2> No

(Do you/ Does anyone in this household 15 years of age and older) use any special aids such as a hearing aid, cane, wheelchair, or some other aid?

<1> Yes ==> Ask "who" if more than one person 15+, and ask 907 for each person identified

<2> No (GO TO 908)

For each person identified in 906, ask:

Which type of aid (do you/does name) use?

<1> Hearing aid

<2> Cane

<3> Wheelchair

<4> Walker

<5> Crutches

<6> Leg brace

<7> Other

(Do you/Does anyone in this household 15 years of age and older) have any difficulty hearing what is said in a normal conversation with another person, (even when using a hearing aid)?

(Note: last parenthetical is filled if 907=1 for anyone in the household)

<1> Yes ==> Ask "who" if more than one person 15+, and ask 909 for each person identified

<2> No (GO TO 910)

For each person identified in 908, ask:

(Are you/Is name) able to hear what is said in a normal conversation at all?

<1> Yes

<2> No

(Do you/ Does anyone in this household 15 year of age and older) have any difficulty lifting and carrying something as heavy as 10 pounds, such as a full bag of groceries?

<1> Yes ==> Ask "who" if more than one person 15+, and ask 911 for each person identified

<2> No (GO TO 912)

For each person identified in 910, ask:

(Are you/Is name) able to lift and carry this much weight at all?

<1> Yes

<2> No

(Do you/does anyone in this household 15 and older) have difficulty walking a quarter of a mile -- about 3 city blocks?

<1> Yes ==> Ask "who" if more than one person 15+, and ask 913 for each person identified

<2> No (GO TO 914)

For each person identified in 912, ask:

(Are you/Is name) able to walk a quarter of a mile at all?

<1> Yes

<2> No

Because of a chronic condition (do you/does anyone in this household 15 and older) need the help of another person with any of the following activities:

Getting in or out of a bed OR a chair? <1> Yes <2> No

Taking a bath OR a shower? <1> Yes <2> No

Doing any household chores such as preparing meals,

OR washing dishes, OR sweeping the floor? <1> Yes <2> No

Going outside the home to shop or

visit the doctor's office? <1> Yes <2> No

[If "yes" to any of the above items, ask "who" and whether they need the help "usually" or "occasionally."]

Final Decision: The recommendation to restructure series to a household-level design was accepted by HHES, with caveats that it depends on resource availability for revising instrument specifications and TMO's/Berkeley's resource availability for revising instrument code. HHES raised concern about using an untested series.

After consulting with TMO staff, DSD decided that the redesigned series cannot be accommodated for 1998. This revision will be considered for 1999, (possibly using a split ballot design). Postponing the revision until 1999 also allows time for cognitive testing of the proposed revised series.

The revision to item 914 is accepted for the 1998 design

8. Health Care Utilization (Adults 15+)

We suggest reordering the series by putting the question on visits to a psychiatric hospital (item 918) after item 915 and asking this of all persons age 15 or over. Responses to that question could be used to fill subsequent questions. (This recommendation was accepted.)

The recommendations below pertain specifically to those questions asked of adults 15+. If these recommendations are accepted, the comparable children's series would need to be modified as well.

Item 915.

Original wording: During the past 12 months, that is, since (date), (was/were) (name/you) a patient in a hospital overnight or longer?

Suggested decision: During the past 12 months, that is, since (date), (was/were) (name/you) admitted to a hospital for an overnight stay or longer?

Justification: Taped interviews indicated that some respondents reported "yes" to item 915, but then reported zero nights in item 919. These people went to the hospital for care but were not admitted for an overnight stay.

Final decision: Suggested wording accepted.

b. Item 916.

Original wording: How many different times (were/was) (you/name) admitted to a hospital for an overnight stay or longer during the past 12 months?

Suggested revision: How many different times (were/was) (you/name) admitted to a (medical/psychiatric/medical or psychiatric) hospital for an overnight stay or longer during the past 12 months?

Justification: With the proposed reordering of the series, this question could be filled based on answers to item 915 and the reordered item 918. See item 918 for additional justification regarding reordering of series.

Final decision: Suggested wording accepted.

c. Item 918.

Original wording: (Was/Were) (name/you) a patient in a psychiatric hospital or a psychiatric unit of a hospital during (this hospital stay/any of your hospital stays in the past 12 months)?

Suggested revision: (Was/Were) (name/you) admitted to a psychiatric hospital or a psychiatric unit of a hospital during the past 12 months?

Note: We recommend moving this item after item 915 and asking it of everyone.

Justification: Interviewing observation reports indicated that the question order in the health care utilization series caused some confusion. One respondent thought the question on number of doctor's visits was only referring to psychiatric visits. She reported "0" to item 921 when, in fact, she'd been to the doctor 6 times. We suggest reordering the series by moving this question on psychiatric hospital stays right after item 915 and use the response to that question to fill subsequent items. This item would be asked of all respondents. We believe that item 915 does not adequately prompt respondents to consider stays in psychiatric hospitals and only those persons who report staying overnight in a hospital are asked item 918 in the current questionnaire.

Final decision: Suggested question wording and item reordering accepted.

d. Item 919.

Original wording: How many total nights did (name/you) spend in a hospital of any type in the past 12 months?

Suggested revision: How many total nights did (name/you) spend in a (medical/psychiatric/medical or psychiatric) hospital in the past 12 months?

Justification: See justification for item 918.

Final decision: Suggested wording accepted

e. Item 921.

Original wording: (Excluding hospital stays,) How many times did (you/name) see or talk to a medical doctor or assistant about (your/his/her) health in the past 12 months?

Instrument problem: Taped interviews indicated the FRs did not read "or talk to" in this question. This is an instrument problem.

Final decision: Instrument will be corrected.

f. Item 923.

Original wording: Is there a place that (name/you) (go/goes) if (you/he/she) (are/is) sick or need advice about (your/his/her) health?

Problems identified: Taped interviews indicated that both respondents and FRs had difficulty with this question. In some cases, respondents would report "no" because the sample person hadn't been to the doctor. In other cases, the FRs are probing inappropriately, for example, "If you were sick , is there a place you could go to for them to take a look at you?"

Final decision: The intention of this question is to measure whether someone has a regular place to go for medical care, not whether they have access to some type of care.

Training issues: The intention of the question needs to be clarified in training and the Fr manual.

g. Item 924.

Original wording: To what kind of place did (you/name) usually go?

<1> Clinic or health center

<2> Doctor's office (or HMO)

<3> Hospital emergency room

<4> Hospital outpatient department

<5> Some other place

Suggested revision: What kind of place is it - a clinic, a doctor's office, an emergency room, or some other place?

Justification: Taped interviews indicated that FRs often read the response categories to the respondents in order to clarify the intent of the question. The suggested wording was agreed upon with HHES but was never incorporated into the pretest instrument. Note that the suggested wording does not include "hospital emergency room" or "hospital outpatient department."

Final decision: Use original wording with the following modifications:
"To what kind of place did (you/name) usually go: a clinic, a doctor's office, an emergency room, or some other place?

FR: IF NECESSARY, READ RESPONSE CATEGORIES

<1> Clinic or health center

<2> Doctor's office (or HMO)

<3> Hospital emergency room

<4> Hospital outpatient department

<5> Some other place (Specify)

h. Item 924A and 924B

Original wording:

924A. During (fill last month) did (you/or anyone in this household) pay any doctor, dentist, or hospital bills, for prescription medicines for (Name)?

(Note: "for" prescription medicines was a typo in the instrument. It should have been "or.")

924B: Not counting amounts reported by another family member or amounts that will be reimbursed by insurance, how much was paid last month for (your/Name's) medical expenses?

Suggested revision:

924A. The next questions are about medical expenses last month, that is, April 1998. Did (you/anyone in this household) have any out-of-pocket expenses for doctor, dentist, or hospital bills for (yourself/name) last month?

(NOTE TO FR: DO NOT INCLUDE PRESCRIPTION MEDICINES, PAYMENTS FOR INSURANCE PREMIUMS, OR EXPENSES THAT WILL BE REIMBURSED BY INSURANCE COMPANIES)

<1> Yes

<2> No (SKIP TO QUESTION ON PRESCRIPTION MEDICINES)

924B. Not counting amounts that will be reimbursed by insurance, how much was paid out-of-pocket for (your/name's) doctor, dentist, or hospital bills last month?

924C. Did (you/anyone in this household) have any out-of-pocket expenses for prescription medicines for (yourself/name) last month?

<1> Yes

<2> No (SKIP TO CK925)

924D. Not counting amounts that will be reimbursed by insurance, how much was paid out-of-pocket for (your/name's) prescription medicines last month?

Justification: FRs indicated that item 924A was too long and that some respondents thought the question was only asking about money spent on prescription medicines (due, in part, to a typo in the instrument). They suggested breaking the question into two separate items. The interviewing observation report indicated that FRs and respondents were unsure whether the amount paid for insurance premiums should be counted. The FR debriefing summary and the taped interviews both highlight the need for a transition statement to indicate that the question was only about last month. Several respondents thought the question was about last year. Taped interviews also show that respondents who had expenses that were covered by their insurance company were unclear how if they should be included.

Final decision: Recommendation accepted with slight modification, as shown below.

924A. The next questions are about medical expenses last month, that is, April 1998. Did (you/anyone in this household) pay any expenses for doctor, dentist, or hospital bills for (yourself/name) last month?

NOTE TO FR: DO NOT INCLUDE PRESCRIPTION MEDICINES, PAYMENTS FOR INSURANCE PREMIUMS, OR EXPENSES THAT WILL BE REIMBURSED BY INSURANCE COMPANIES.

NOTE: "PAY" REFERS TO "OUT-OF-POCKET " EXPENSES.

<1> Yes

<2> No (SKIP TO QUESTION ON PRESCRIPTION MEDICINES)

924B. Not counting amounts that will be reimbursed by insurance, how much was paid for (your/name's) doctor, dentist, or hospital bills last month?

924C. Did (you/anyone in this household) pay any expenses for prescription medicines for (yourself/name) last month?

<1> Yes

<2> No (SKIP TO CK925)

924D. Not counting amounts that will be reimbursed by insurance, how much was paid for (your/name's) prescription medicines last month?

9. Health Insurance

a. Item 950.

Original wording: The next few questions are about health insurance coverage in 1996.

This is a list of different types of health insurance coverage. I'd like to know if (you/anyone in this household) was covered by the following types of health insurance at ANY TIME from January through December 1996:

a. Medicare, the government medical plan for persons 65 and over and for persons with disabilities

b. Medicaid or (state program name), the government medical plan for persons with low incomes

c. CHAMPUS/TRICARE, CHAMPVA, Military health, Indian Health Service, or any other government- provided health insurance plan including (fill local name.)

d. A plan provided (by a person in this household) through a current or past employer or union

e. A plan purchased directly from an insurance company, that is, a private plan not related to a current or past employer

f. A plan of someone not living in this household

Suggested revision: FLASHCARD R.

[Ask for each person in the household. Fill "The next...1997" for first person in the household only. ]

(The next few questions are about health insurance coverage in 1997.) Which type of health insurance, if any, (was/were) (you/name) covered by at any time between January and December 1997?

MARK ALL THAT APPLY

a. Medicare, or other health insurance paid for by Medicare (for persons over 65 or persons with disabilities)

b. Medicaid or (state name) (for persons with low incomes)

c. CHAMPUS/TRICARE, CHAMPVA, Military health, Indian Health Service, or any other government- provided health insurance plan (including (fill local name))

d. A plan provided (by a person in this household) through a current or past employer or union
e. A plan purchased directly from an insurance company, that is, a private plan not related to a current or past employer

f. A plan of someone not living in this household

g. Not covered by health insurance at all during (year)

Justification: Taped interviews indicated that both respondents and FRs lost sight of the fact that this question is asking about the past calendar year. Asking the question about each person in the household individually and including the reference period at the end of the question should reduce the likelihood of respondents erroneously reporting current coverage. (If this recommendation is implemented, questions asking who was covered by the different types of insurance marked in item 950 can be deleted. This includes items 951A, 953A, 955A, 957A, 959A, 961A.)

Taped interviews also indicated that Medicare recipients who are enrolled in an HMO that is paid for by Medicare are unclear how to respond to this question. We suggest modifying category "a" as indicated above to alleviate the confusion. Additional coverage carried by some Medicare recipients, often referred to as "Medi-gap," is considered a plan purchased directly from the insurance company. Rather than modifying the flashcard to include this term, we suggest that FRs receive additional training regarding this type of health insurance and how to adequately probe Medicare recipients for this information.

The flashcard and instrument screen should contain the lettered response category. The pretest flashcard contained lettered response options and the instrument screen contained a yes/no format for each response category without the preceding letter. The taped interviews indicated that respondents would say "d" and the FR wouldn't know to which category they were referring since "d" did not appear on their screen.

Final decision: Although HHES accepted this recommendation, it cannot be implemented in 1998 due to resource constraints within TMO, but will be held for implementation in 1999.

It was also agreed that the response categories should be reordered as shown below, so the more common types of insurance are towards the top of the list. (Along with the reordering, some revisions have also been made in the use of bold type.) It is unclear at this time if the reordering of the insurance types is possible for 1998, or if that also needs to be held until 1999.

The letters used on the flashcard should be added to the FRs screen.

a. A plan provided (by a person in this household) through a current or past employer or union

b. A plan purchased directly from an insurance company, that is, a private plan not related to a current or past employer (e.g. "Medi-gap")

c. Medicare, or other health plan paid for by Medicare (for persons over 65 or persons with disabilities)

d. Medicaid (or state name) (for persons with low incomes)

e. CHAMPUS/TRICARE, CHAMPVA, Military Health

f. Indian Health Service

g. Other government-provided health insurance plan (including state name)

h. A plan of someone not living in this household

i. Not covered by any kind of health insurance for the entire year

Note: FRs will need to write in the state names on their flashcards, if appropriate, since printing flashcards tailored to each state/local program is not possible. Also, not that the parenthetical "(e.g. Medi-gap)" for option b is to be shown on the FR screen but not on the flashcard.

Training Issue: We suggest FRs receive additional training regarding Medi-gap coverage and how to adequately probe Medicare receipents for this information.

b. Item 951B (and other items asking for months covered)

Original wording: During which months in 1996 (was/were) (name/you) covered by Medicare?

Suggested revision: Were you covered by (type of plan) for all of 1997 or for only part of 1997?

<1> All year

<2> Part of year (ASK WHICH MONTHS)

Which months were you covered by (type of plan) in 1997?

<1> Jan <7> July

<2> Feb <8> Aug

<3> Mar <9> Sep

<4> Apr <10> Oct

<5> May <11> Nov

<6> Jun <12> Dec

[blind] <D,R>

Note: This suggestion also applies to 953B, 955C, 957D, 959C, 961B, 966.

Justification: Taped interviews indicate that FRs will read the question as worded and then probe using wording similar to that included in the suggested wording above. Additionally, taped interviews indicated that most respondents are covered by the same plan all year.

Final decision: Use suggested wording. Change this item as well as 953B, 955C, 957D, 959C, 961B, and 966.

c. Item 955C.

Original wording: During which months in 1996 (was/were) (name/you) covered by this type of health insurance?

Suggested revision: During which months in 1997 (was/were) (name/you) covered by (fill from 955B)? [Fills from 955B are: CHAMPUS, CHAMPVA, Military health, Indian Health Service, a government-provided plan (including (state name))]

Note: This change also applies to items 957D and 961B.

Justification: The interviewing observation report suggested including the name of the plan rather than using the generic phrase "this type of health insurance."

Final decision: Use suggested wording. Change this item as well as 957D and 961B.

d. Item 957A.

Original wording: Who was covered by an employer or union provided plan?

Suggested revision: Who was covered by an employer or union provided plan in 1997?

Justification: Taped interviews indicated that some respondents reported persons currently covered rather than those covered during the past calendar year. Similar questions about other types of health insurance include the reference period, suggesting that this was an oversight when the questionnaire was developed.

Final decision: Suggestion is accepted.

e. Item 957B.

Original wording: Which person in this household is the policy holder of (name)'s plan?

Suggested revision: Which person in this household was the policy holder of (name's) plan during 1997?

Justification: The current wording sounds like we are asking about current coverage, but the question is intended for coverage during the past calendar year. Taped interviews indicated that respondents, and in some cases FRs, didn't know the question was about last year. We suggest adding the reference period to the end of the question to clarify this.

Final decision: Use suggested wording.

Instrument problem: Taped interviews indicated that items 957B, 957C, and 957D were not asked about children under 15. These questions should be asked about ALL household members as appropriate.

f. Item 957C.

Original wording: Does the employer or union pay for all, part, or none of the cost of the plan?

Suggested revision: Did the employer or union pay for all, part, or none of the cost of the plan in 1997?

Justification: Original wording asks about current coverage. Suggested wording asks about past calendar year, which is the concept of interest.

Final decision: Use suggested wording.

g. Item 963.

Instrument problem: Responses of "don't know" or "refused" should skip to 968, not 967.

Final decision: Instrument will be corrected

h. Item 965.

Original wording: FLASHCARD R.

What type of health insurance (was/were) (name/you) covered by in 1996?

{SHOW FLASH CARD; MARK ALL THAT APPLY}

<1> Medicare - the government plan for persons 65 and over and for persons with disabilities

<2> Medicaid or (fill local name) -- the government medical plan for persons with low incomes

<3> CHAMPUS/TRICARE/CHAMPVA, Military health, Indian health service, or any other government-provided health insurance plan, including (fill local name)

<4> A plan provided (by a person in this household ) through a current or past employer or union.

<5> A plan purchased directly from an insurance company, that is, a private plan not related to a current or past employer.

<6> A plan of someone not living in this household.

Suggested revision: Keep question wording the same. Use flashcard for item 970 shown below in item 968. Add wording to medicare response category on flashcard, as suggested above in item 950.

Justification: This item is asked of persons who are erroneously reported as not having health insurance in the previous calendar year. The suggested wording for item 950 would alleviate the need for this question since it collects health insurance at the person level rather than the household level. If the suggested change for item 950 is not accepted, we suggest using the same flashcard for this item and item 970. FRs noticed that the flashcards for these two items were similar but not identical. They suggested making them the same and eliminating one of the flashcards. We suggest using Flashcard T because it is easier to read and includes the distinction between policyholder and dependents. The FRs also requested that the local name for government-provided health insurance be included on the screen and on the flashcard. In the pretest instrument, the FRs were required to specify the name of the local government provided insurance (e.g. MinnesotaCare). The fill for the local name appears in number 3 on Flashcard R. If the recommendation to substitute Flashcard T for this item is accepted, the local name should be included on Flashcard T and on the instrument screen (the specs shows no fill for the local name) so that FRs don't have to type this information into the "specify" screen.

Final recommendation: In 1999 this item will be deleted. In 1998, keep the question wording the same, but replace the response options with those from Flashcard T (shown below under item 968.)

i. Item 967

Original wording: Which answer on this card best describes the reason why (name/you) (wasn't/weren't) covered by health insurance in 1996?

MARK ALL THAT APPLY

<1> Job layoff, job loss, or any reasons related to unemployment

<2> Employer does not offer health insurance

<3> Can't obtain health insurance because of poor health, illness, or age

<4> Too expensive; can't afford health insurance

<5> Don't believe in health insurance

<6> Have been healthy; not much sickness in the family; haven't needed health insurance

<7> Able to go to VA or military hospital for medical care

<8> Person outside this household did not provide health insurance

<9> Other (specify)

Suggested revision: Keep wording the same. Add a response category "Not eligible because haven't worked at job long enough yet."

Justification: An FR suggested this category be added to accommodate people who are working but haven't been at a job long enough to be eligible for health insurance. If this is a group of people that are of interest to analysts, then such a category should be added. Otherwise, existing categories can remain as they are.

Instrument problem: The question asks for one answer, while the instructions to the FR asks for as many as apply. HHES wants this to be a "mark all that apply" question.

Final decision: Since the question is intended to be a "mark all that apply", the question needs to be slightly modified to indicate that multiple

responses are acceptable. Also, HHES requested that the same response categories used in SIPP be included in SPD, as shown below.

What answers on this card best describe the reasons why (name wasn't/you weren't) covered by health insurance in 1997?

Probe: Any other reasons?

<1> Too expensive; can't afford health insurance.

<2> No health insurance offered by employer of self, spouse, or parent

<3> Not working at a job long enough to qualify

<4> Job layoff, job loss, or any reason related to unemployment

<5> Not eligible because working part time or temporary job

<6> Can't obtain insurance because of poor health, illness, age, or pre-existing condition

<7> Dissatisfied with previous insurance OR don't believe in insurance

<8> Have been healthy; not much sickness in family; haven't needed health insurance

<9> Able to go to VA or military hospital for medical care

<10> Covered by some other health plan, such as Medicaid

<11> No longer covered by parents' policy

<12> Other

j. Item 968

Original wording: These next few questions are about CURRENT health insurance coverage. (Are/Is) (you/anyone in this household) CURRENTLY covered by any type of health insurance, including Medicare and Medicaid?

Suggested revision:

[For persons covered either all year or at the end of 1997, ask the following question:]

968a. (Is name/Are you) CURRENTLY covered by (fill types of insurance this person reported having in Devember 1997)?

<1> Yes (Ask 968b)

<2> No (ASK FOR CURRENT COVERAGE)

FLASHCARD T

968b. What type of health insurance, if any, (are you/is name) currently covered by?

<1> Employer/union provided (policy holder)

<2> Employer union provided (dependent)

<3> Purchased plan directly--not employer/union, (e.g. medi-gap) (policy holder)

<4> Purchased plan directly--not employer/union (e.g. medi-gap) (dependent)

<5> Medicare, or other health plan paid for by Medicare

<6> Medicaid, or (fill state name)

<7> CHAMPUS/TRICARE/CHAMPVA

<8> Military Health

<9> Indian Health Service

<10> Other government health insurance plan or (fill local name) (specify)

<11> Covered by someone outside this household

<12> Not currently covered by health insurance

[blind] <D> Don't know

[blind] <R> Refused

[For persons not covered at all during previous calendar year or for persons not covered in December of previous calendar year, ask the following question(s):]

969a. (Is/Are) (name/you) currently covered by health insurance?

<1> Yes (Ask 969b)

<2> No (Skip out of series)

FLASHCARD T

969b. What type of health insurance (are/is) (you/name) currently covered by?

(Same categories as above)

Justification: Taped interviews, interviewing observation reports, and the FR debriefings indicated that there was confusion caused by the shifting reference periods in the health insurance series. The questions about current coverage sounded very similar to the questions about coverage during the past calendar year. In addition, many respondents and FRs thought the questions on the past calendar year were actually about current coverage (see previous comments). Taped interviews indicate that for many people health insurance does not change from one year to the next. Asking if the health insurance is the same as that reported for last year simplifies this series for the vast majority of people. Additionally, taped interviews and observation reports indicated that the original wording mislead respondents into thinking we were asking if they were covered by Medicare or Medicaid. They would report "no" when, in fact, they were currently covered. This required FRs to probe extensively to obtain the correct information. The suggested wording is similar to that contained in the March CPS.

Final decision: The suggestion was accepted by HHES but cannot be implemented for 1998 due to TMO resource constraints. This revision will be held for 1999 implementation. As an alternative option for 1998 implementation, we propose the following strategy be adopted for collecting current coverage. This has not yet been discussed with HHES or DSD, so it is unknown whether the suggested revision can be implemented for the 1998 SPD.

Alternative suggestion for 1998:

[For persons covered either all year or at the end of 1997, ask the following question:

968a. (Is everyone in this household/Are you) CURRENTLY covered by the same type(s) of insurance (you/they) had in December 1997?

[Display insurance type(s) covered by in December 1997. If possible, when persons in hhld covered by different types, display type by persons covered]]

<1> Yes (end series)

<2> No (Ask 968b)

968b. Who is CURRENTLY not covered by the same type of insurance they had in December 1997?

For each person identified in 968b, ask:

968c. What type of health insurance, if any, (are you/is name) currently covered by?

<1> Employer/union provided (policy holder)

<2> Employer union provided (dependent)

<3> Purchased plan directly--not employer/union, (e.g. medi-gap) (policy holder)

<4> Purchased plan directly--not employer/union, (e.g. medi-gap) (dependent)

<5> Medicare, or other health plan paid for by Medicare

<6> Medicaid, or (fill state name)

<7> CHAMPUS/TRICARE/CHAMPVA

<8> Military Health

<9> Indian Health Service

<10> Other government health insurance plan or (fill local name) (specify)

<11> Covered by someone outside this household

<12> Not currently covered by health insurance

[blind] <D> Don't know

[blind] <R> Refused

10. Food security

Problem identified: During the FR debriefings, many FR's indicated that they felt embarrassed asking these questions in higher income households. The observation reports also indicated that these questions seemed unnecessary to ask in upper income households. The CPS Food Security Supplement from which these questions were taken uses an income screener. Households are still asked item 1000 (quantity and quality of food eaten in the household) and one other item (not included in this set). Depending on the answer to these two items, persons in households showing no signs of food insecurity are skipped over the remaining series of questions. The income screener used in CPS is based on a categorical variable for family income in the previous 12 months. We suggest considering the use of an income screener with these items.

Final decision: FNS has agreed to use an income screener for the food security questions. If the family income exceeds 185 percent poverty and the respondent indicates that the household has "enough food to eat and the kinds of food they want" (response <1> in item 1000), then the remainder of the food security questions can be skipped. The CPS Food Security Supplement contains a similar economic screener. Specifications are available from Jenny Hess in CSMR or the CPS Branch in DSD. POP division has expressed concerns regarding whether this change can be made in the required time frame. POP, TMO, and DSD need to discuss this issue and determine what priority should be given to this requested change.

a. Item 1000.

Original wording: These next questions are about the food eaten in your household. Which of these statements best describes the food eaten in your household in the last 12 months -- (I/we) have enough to eat and the kinds of food (I/we) want, (I/we) have enough to eat but not always the kinds of food (I/we) want, sometimes (I/we) don't have enough to eat, or often (I/we) don't have enough to eat?

Suggested revision: Keep wording the same. Include a flashcard for this item.

Final decision: The Food and Nutrition Service (FNS) of the US Department of Agriculture (USDA) requested that the lead-in to this question be changed as follows:

Flashcard X.

These next questions are about the food eaten in your household in the last 12 months, since May 1997, and whether you were able to get the food you need. Which of these statements best describes the food eaten in your household in the last 12 months -- (I/we) have enough to eat and the kinds of food (I/we) want, (I/we) have enough to eat but not always the kinds of food (I/we) want, sometimes (I/we) don't have enough to eat, or often (I/we) don't have enough to eat?

Replace the current lead-in with this new one and include a flashcard with the answer categories.

Training issues: FRs need a better understanding of why the food security series is included in the SPD. This could reduce the FRs' discomfort asking these questions and better enable the FRs to respond to concerns that respondents raise. The FR manual and training materials from the CPS Food Security Supplement should provide some of this information. FNS may be able to advise us on additional materials to include in training and the manual.

11. Children's school enrollment

a. Item 1103 (For children 3-6)

Original wording: Since September 1996 was (Name) enrolled in preschool? (Note: Include pre-kindergarten as well as preschool.)

Suggested revision: At any time between September 1997 and April 1998 was (Name) enrolled in preschool?

Justification: The suggested wording is based on the results of cognitive interviews, and was previously agreed upon. However, the wording was not included in the pretest instrument.

Final decision: Use suggested wording.

b. Item 1108 (For children 5-6)

Original Wording: At anytime since September 1996 was (name) enrolled in school?

Suggested revision: At anytime between September 1997 and April 1998 was (name) enrolled in school?

Justification: The suggested wording is based on the results of cognitive interviews, and was previously agreed upon. However, the wording was not included in the pretest instrument.

Final decision: Use suggested wording.

c. Item 1108A

Original wording: Since September 1996 which months was (name) enrolled in school?

Suggested revision: Between September 1997 and April 1998 which months was (name) enrolled in school?

(Note: Add a category for >all' months).

Justification: The suggested wording is based on the results of cognitive interviews, and was previously agreed upon. However, the wording was not included in the pretest instrument.

Final decision: Use suggested wording.

d. Item 1109

Original wording: Since September 1996 what was the highest grade in which (child name) was enrolled?

Suggested revision: In what grade was (name) enrolled in April 1998?

Justification: The suggested wording was previously agreed to based on cognitive interviewing and should help to alleviate confusion generated by the original question. In the FR debriefings, FRs noted that respondents had problems knowing what was meant by the 'highest grade in which ... was enrolled'. This was confirmed by the review of the taped interviews which indicated that respondents had a difficult time with the highest grade concept. These problems seemed to stem in part from the reference period.

Final decision: Use suggested wording. For persons who dropped out of school before April 1998, fill with the last month they were enrolled in school.

Training issues: FRs should be trained how to handle situations in which a high school student simultaneously takes college classes.

e. Item 1116

Original wording: Did (name) attend special classes for gifted students or do advanced work in any subjects since September 1996?

Suggested revision: Did (name) attend classes for gifted students or do advanced work in any subjects between September 1997 and April 1998?

Justification: Review of taped interviews indicate that some respondents seemed to focus on 'special' rather than 'gifted or advanced work'. It is possible that respondents are accustom to hearing the term special used in the context of special or remedial education and do not think of "special" as describing gifted and talented or advanced course work. Eliminating the word "special" should alleviate this problem.

Final decision: Use suggested wording.

Training issues: FRs and respondents need clarification regarding what constitutes "classes for gifted students" and "advanced work." Kurt Bauman (POP) provided the following clarification.
"The central idea of this question is to identify students who are taking ACADEMICALLY-ORIENTED courses covering subject matter that goes ABOVE AND BEYOND THE STANDARD CURRICULUM for the level of school they are in. Advanced placement (AP) courses are an obvious example of this. Students who are in "high track" courses qualify, so long as the track involves advanced academic subjects or leads directly to courses that do. DO NOT COUNT special "enrichment" courses (e.g. nature and the environment, art workshops, music, performance or internships) unless they are designed for gifted students and prepare them for advanced ACADEMIC course work later on. Any class that is remedial in nature, is designed to help students catch up in certain subject areas or deal with learning or behavioral challenges should not be included."

f. Item 1117

Original wording: Was (Name) suspended or expelled from school at any time since September 1996?

Suggested revision: Between September 1997 and April 1998 was (Name) suspended or expelled from school?

Justification: The suggested wording was previously agreed upon by CSMR/POP/DSD in order to have consistency in reference periods between items in this series.

Final decision: Use suggested wording.

g. Item 1119

Original wording: Since September 1996 did (Name) change schools?

Suggested revision: Between September 1997 and April 1998 did (Name) change schools?

Justification: The suggested wording was previously agreed upon by CSMR/POP/DSD in order to have consistency in reference periods between items in this series.

Final decision: Use suggested wording.

h. Item 1120

Original wording: How many times did (Name) change schools Since September 1996?

Suggested revision: Since September 1997 how many times did (Name) change schools?

Justification: The suggested wording was previously agreed upon by CSMR/POP/DSD in order to have consistency in reference periods between items in this series.

Final decision: How many times did (name) change schools?

Children's enrichment activities

The primary problem with this series, identified through review of the taped interviews, was a failure by FRs to read the questions as worded for this module. In some cases, major parts of the questions were not read to respondents. (These problems can be handled through FR training.) Other problems identified through the taped interviews were related to reference periods.

a. Item 1124

Original wording: Since September 1996 was (Name) on any kind of a sports team?

Suggested revision: Between September 1997 and April 1998 was (Name) on any kind of sports team?

Justification: The suggested wording was previously agreed upon by CSMR/POP/DSD in order to have consistency in the reference period used for the school enrollment series.

Final decision: Use suggested wording.

b. Item 1125

Original wording: Did (name) take lessons after school or on weekends in activities such as music, dance, language, or karate at any time since September 1997?

Suggested revision: Did (name) take any kind of lessons after school or on weekends in activities such as music, dance, language, or karate at any time between September 1997 and April 1998?

Justification: The suggested wording was previously agreed to based on cognitive interviewing, but was not included in the pretest instrument.

Final decision: Use suggested wording.

c. Item 1126

Original wording: Did (name) participate in any clubs or organizations after school or on weekends, such as Scouts, school newspaper, (Boys/Girls) club, or a religious group at any time since September 1996?

Suggested revision Did (name) participate in any clubs or organizations after school or on weekends, such as Scouts, school newspaper, (Boys/Girls) club, or a religious group at any time between September 1997 and April 1998?

Justification: The suggested wording was previously agreed upon by CSMR/POP/DSD in order to have consistency in the reference period used for the school enrollment series.

Final decision: Use suggested wording.

d. Item 1127

Original wording: The next few questions are about television viewing.

Are there family rules about what programs (name) can watch?

Suggested revision: Are there family rules about how much television or what programs (name) can watch?

Justification: The suggested wording was previously agreed to based on cognitive interviewing, but was not included in the pretest instrument.

Final decision: Use suggested wording.

e. Item 1128

Original wording: How many hours per week does (name) usually watch television?

INCLUDE BOTH VIDEOS AND TV VIEWING

Suggested revision: Including weekends, how many hours per week does (name) usually watch television?

INCLUDE BOTH VIDEOS AND TV VIEWING

Justification: The suggested wording was previously agreed to based on cognitive interviewing, but was not included in the pretest instrument.

Final decision: Use suggested wording.

f. Item 1132

Original wording: During the past month, did you (or any family member) take (name) on any kind of outing such as to a park, library, zoo, church, playground, or shopping center (Read categories)

Suggested revision: How often in the past month did you (or any family member) take (name) on any kind of outing such as to a park, library, zoo, church, playground, or shopping center - never, once in the past month, about once a week, several times a week, every day or almost every day?

Justification: The suggested wording was previously agreed to based on cognitive interviewing, but was not included in the pretest instrument. In some cases respondents asked the FR to repeat the reference period. Implementing this wording should reduce reference period problems identified through taped interviews.

Final decision: Use suggested wording.

13. Children's disability

We suggest having one set of children's disability questions to be asked of all children age 0 to 14 as appropriate. HHES accepts this recommendation; however, due to resource and time constraints, the recommendation will not be included in the 1998 SPD instrument, but will be considered for the 1999 SPD. This set of questions will be developed at a later date and are not included here due to time constraints. When developing these questions, we need to consider how this will work in households in which there is more than one designated parent, since the children's questions are supposed to be asked of the designated parent and this proposal would violate that principle.

14. Children's health care utilization

a. Item 1210.

Original wording: During the past 12 months, that is since (date), was (name) a patient in a hospital overnight or longer?

Suggested revision: During the past 12 months, that is, since (date), was (name) admitted to a hospital for an overnight stay or longer?

Justification: This change was made to the comparable item in the adult section. See justification under item 915 in adult health care utilization above.

Final decision: Suggested wording accepted.

b. Item 1220.

Original wording: To what kind of place did (name) usually go?

<1> Clinic or health center

<2> Doctor's office (or HMO)

<3> Hospital emergency room

<4> Hospital outpatient department

<5> Some other place

Note: We are not sure what wording was actually used in the pretest instrument. To make the adult and children's items comparable, we suggest using the agreed upon wording for the adult item, which is included below under "final decision."

Suggested revision: What kind of place is it - a clinic, a doctor's office, an emergency room, or some other place?

Justification: Same change was suggested to comparable adult item. See justification for item 924 in adult health care utilization.

Final decision: Use original wording with the following modifications:
"To what kind of place did (you/name) usually go: a clinic, a doctor's office, an emergency room, or some other place?

FR: IF NECESSARY, READ RESPONSE CATEGORIES

<1> Clinic or health center

<2> Doctor's office (or HMO)

<3> Hospital emergency room

<4> Hospital outpatient department

<5> Some other place (Specify)

c. Item 1221a and 1221b.

Original wording 1221a: During (fill last month) did (you/or anyone in this household) pay any medical expenses for (name)--include any doctor, dentist, or hospital bills, or any prescription medicines?

Note: This wording is different than the comparable question in the adult section. We recommend using the agreed upon wording for the adult question as shown below under "final decisions."

Original wording 1221b: Not counting amounts that will be reimbursed by insurance, how much was paid last month for (name's) medical expenses?

Suggested revision 1221a. The next questions are about medical expenses last month, that is, April 1998. Did (you/anyone in this household) have any out-of-pocket expenses for doctor, dentist, or hospital bills for (yourself/name) last month?

(NOTE TO FR: DO NOT INCLUDE PRESCRIPTION MEDICINES, PAYMENTS FOR INSURANCE PREMIUMS, OR EXPENSES THAT WILL BE REIMBURSED BY INSURANCE COMPANIES)

<1> Yes

<2> No (SKIP TO QUESTION ON PRESCRIPTION MEDICINES)

1221b. Not counting amounts that will be reimbursed by insurance, how much was paid out-of-pocket for (your/name's) doctor, dentist, or hospital bills last month?

1221c. Did (you/anyone in this household) have any out-of-pocket expenses for prescription medicines for (yourself/name) last month?

<1> Yes

<2> No (SKIP TO CK1280)

1221d. Not counting amounts that will be reimbursed by insurance, how much was paid out-of-pocket for (your/name's) prescription medicines last month?

Justification: This change was suggested for the comparable adult items. See the justification for items 924A and 924B in the adult health care utilization section.

Final decision: Recommendation accepted with slight