Back to contents index

Chapter 3

The post-enumeration survey

A post-enumeration survey (PES) is undertaken to determine the degree of undercount or overcount in a population census, and to evaluate the quality of data collected during the census. In Census ’96, this exercise was undertaken shortly after the enumeration phase of the census, during November and December 1996.

In line with the procedure followed during the census, the PES involved seeking to identify the boundaries of each EA in the sample, to list all the visiting points within the enumerator area (EA), and to administer a questionnaire in each household at each visiting point. The PES questionnaire covered particular details of the members of the household as a check upon the accuracy of the census information; but it also asked about who had been reached or missed.

Conducting the PES involved the following stages:

• Preparation, consisting of drawing a sample, questionnaire development, and staffing and training.

• Fieldwork, consisting of re-identification of EA boundaries, re-listing of visiting points, questionnaire administration and completion of a new census summary book.

• Arriving at the preliminary estimates of the extent of undercount.

• Matching, data capture and final calculations.

A sample of 800 EAs (approximately 1% of the total number of EAs in the country) was drawn for the PES in the following manner:

• The country was stratified by province, as well as by type of EA (formal or informal urban areas, commercial farms, tribal or other non-urban areas).

• Independent systematic samples of EAs were then drawn for each stratum within each province.

• The sampling frame that was used for the PES was constructed from the preliminary database of EAs which was established during the demarcation and listing phase of the census. Empty EAs (consisting of unoccupied land), and nearly empty EAs were excluded from the sample.

The questionnaire used for the PES was a much shorter version of the census questionnaire. It included questions on age, gender, marital status, population group, home language and level of education of each household member.

As noted earlier, respondents were also asked whether or not each individual in a particular household had been counted during the census and, if so, whether they were counted in that specific household or elsewhere.

Altogether, approximately 1 850 temporary staff, including 1 600 fieldworkers or enumerators (two in each EA), 200 fieldwork supervisors and 50 regional managers, were employed to work on the PES.

Most of the staff were selected from existing census workers who had been recommended by their managers as being highly competent in carrying out their enumeration tasks. To ensure that they did not evaluate their own previous work, PES staff were not allocated to work in the same EA as those in which they had worked in the census.

Regional managers for the PES were trained by Stats SA head-office staff. The regional managers in turn trained the fieldwork supervisors and fieldworkers at regional level. Training manuals were developed for all three levels of staff. The aim was to achieve more uniform and reliable enumeration in the PES than had been possible in the census.

During the fieldwork phase of the PES, information was obtained on the number of households in a particular EA and on the number of individuals in each household and also on the number of visiting points.

The fieldworkers were required to work in pairs for the tasks of re-identification of the boundaries of each EA in the sample and re-listing of the visiting points, and for questionnaire administration.

• As a first task, the fieldwork pair was instructed to check the boundaries of the EA which had been allocated to them. New summary books (PES summary books), containing only EA descriptions and maps where available, or aerial photographs, were supplied for the task.

• As their second task, the fieldworkers were required to list afresh all the visiting points in the EA to which they had been assigned, without having access to the previous listings.

• They were then required to administer a questionnaire in each household in the EA to which they had been assigned.

• Finally, the PES summary book was completed in the same way as in the census. After enumeration, a fieldworker was required to enter information on the number of males and females, and the total number of people in each household, at every visiting point. He or she was also required to indicate all vacant premises or stands; premises which contained non-residential buildings, such as churches, schools, and business premises; those who refused to be interviewed; and failures to make contact with specified households in each EA.

Preliminary estimates to determine the extent of undercount relied on the respondents and what they said about whether or not the household itself, and each person in the household, had been counted during the census. A brief description of how the preliminary estimates were calculated is given in Chapter 4. For a more detailed description, the reader is referred to Census ’96: preliminary estimates of the size of the population of South Africa (Central Statistical Service, 1997).

The following is a brief description of the way in which matching was done. For a more detailed description, the reader may want to consult Calculating the undercount in Census ’96 (Statistics South Africa, 1998), on which this section is based.

The objective of the matching exercise was to compare the PES questionnaires with their equivalents in the census, firstly household by household, and secondly individual-by-individual.

The process involved the following steps:

• Matching of EAs: This task involved finding the corresponding census EA descriptions, maps, aerial photographs, etc. in the census summary book and seeing whether or not they matched the descriptions, maps, etc., in the PES summary book. In some instances, it was not possible to find a census EA corresponding the PES EA. Where boundaries appeared to differ, it was necessary to visit the province to resolve the problems and to find census questionnaires from adjacent EAs. When boundaries were unclear, this necessitated a visit to the actual EA in field to obtain more clarity.

• Matching of households: Initially the addresses or other identifiers in the PES summary books were compared to the addresses or other identifiers in the census summary books. This was straightforward in the case of formal areas with known, precise addresses, but it was more difficult in the case of informal settlements and farms, without precise addresses. Once the matching of listings in the summary books was completed, the questionnaires for corresponding PES and census households were extracted and compared. These questionnaires were found by comparing addresses or other identifiers in the summary books.

• Matching of persons: Using the questionnaire/s for a particular household, an attempt was made to match each person in the PES questionnaire against each person in the equivalent census questionnaire. Where names were given, this process was relatively easy, but people were asked to give only first names or initials in both processes. Where people could not be matched by name, demographic information such as age, gender and level of schooling was used.

As far as households are concerned, the results of the matching process involved one of the following:

• A matched outcome.

• A clearly missed result, for example, where there was a failure to list a visiting point, or a refusal or non-contact in the census but not in the PES.

• An unresolved case, because there was insufficient information to conclude whether the households were matched or missed, that then required statistical imputation.

Regarding individuals, in the matching process there could have been:

• A matched outcome, because there was clear evidence the person was enumerated during the census at the PES address.

• A missed result, where the person was excluded, even though the other members of the household were clearly matched.

• A missed result, because the household was missed altogether.

• An unresolved case, because there was insufficient information to conclude whether the person had been matched or missed.

Once the matching process of households and individuals was completed, the matching data had to be merged with the original PES questionnaire data, which had already been captured by a sub-contractor, as previously discussed.

The data from the merged data sets were used to calculate the extent of undercount and the adjustment factors for the new, final population estimates. The methodology is briefly described in Chapter 5.

The conducting of a PES was a strategy adopted by Stats SA to enable it to move away from long-range demographic modelling and its sensitive assumptions as a basis for adjusting the census. As the first nationwide PES conducted by the official statistics agency, and planned at short notice, it was a powerful exercise, and the careful matching allowed for greater precision in the undercount adjustment. But it had certain limitations.

• Methodologies and procedures were tried out and revised throughout the process.

• The questionnaire design did not make adequate provision for those households and individuals who had moved into a new EA after the census.

• The exclusion of the full names of people on both the census and the PES questionnaires, to enhance confidentiality in politically tense areas, made the matching process difficult.

• The matching process was especially difficult in those areas where there were no addresses.

• Unclear EA boundaries, resulting in the inability to find the matching questionnaires, were the main reason for the unresolved cases (22%) that then required statistical imputation.

• Hard-to-reach people may have been missed in both the PES and the census.

• The PES could not measure all incidents of double counting.

• It was more difficult to confirm a definite miss than a definite match, so it is likely that there was some bias towards setting cases to unresolved.

• The processes of matching and data-entry can be improved.

Despite these problems, a very high rate of matching was achieved, and it was thus possible to move beyond relying on the yes/no responses to determine undercount rates. The final calculations discussed in Chapter 5, based on the empirical evidence of the PES adjustments, are thus likely to be a reliable indication of the extent of undercount in the 1996 population census.