Comparing household & telephone survey samples: a case study of the 1998 Smoking & Health Survey

Daniella Germain

Sarah Durkin

CBRC Research Paper Series No. 34 

To address issues of data comparability resulting from a change in data collection methods for the Victorian Smoking and Health population survey, data collected in 1998 that used both household and telephone survey methods were examined.

Un-weighted survey data indicated that household and telephone survey samples varied significantly in relation to their distribution across most socio-economic groups. The household survey sample over-represented those with a higher socio-economic status, while the telephone survey sample was more highly educated than household survey respondents. Smoking rates were also found to be significantly higher among household survey respondents.

To adjust for demographic differences, data from both surveys were weighted by age, sex and socio-economic status (SEIFA). Differences found between education groups across the two samples remained; however, regular smoking rates between the two samples were no longer significantly different when weighted.

Multivariate analyses also found that there was no effect of survey type on regular smoking when controlling for demographic influences. Further, for all but one demographic sub-group (30-49 year olds), interactions between demographic sub-groups and survey type were not significant.

The current study suggests that comparing these two survey methods achieves similar results in terms of smoking behaviour when survey data are weighted to overall population estimates of age, sex and SEIFA. Findings indicate that when survey data is appropriately weighted, and the same survey form is used, it is statistically valid to compare data collected from household and telephone survey methodologies.