The Effects of Methods on Results in Dialectology

Although dialectology was among the first disciplines to use the survey as a research tool, dialectologists, unlike researchers in other social sciences, have done little work in assessing the effects of their survey methods on their results. This paper attempts to begin a dialog on the effects of methods on results in dialectology by comparing results from five surveys which overlap both in their geographic coverage and in some of the linguistic information they elicit. The surveys differ in their methods of administration (face-to-face interviews vs. telephone interviews), sample construction (purposive vs. random), and the kinds of data they elicit (observations of behavior vs. self-reports). Our comparison of the different surveys shows that while different modes of administration have little effect on results, even slight differences in survey populations affect results significantly, as does the kind of data elicited. Surprisingly, self-reports seem to reflect the linguistic behavior of a population for some features better than observations of behavior do. The effects of the type of sample used are not clear from this study, although random samples have the advantage of explicitly accounting for sampling error and allow for a wide range of inferential statistics that cannot be used with purposive samples. Finally, the comparison suggests that there is no single 'best' type of survey. Different research problems require different kinds of surveys. What is important is that samples not be constructed in haphazard ways and that we explicitly take into account the effects of our methods on our results.