Limitations of Questionnaires and Web Experiments

Web-based questionnaires and questionnaires are a vital epidemiologic technique that provides crucial information on the condition of public health and disease. They are a common means of gathering data that is often less costly and time-consuming than face-to-face interviews, mail-in questionnaires, or automated telephone menu systems. However questionnaires, surveys and Web experiments are not without limitations that must be addressed in order to ensure the validity and reliability of results.

A questionnaire can be influenced by response bias, the tendency of respondents to answer questions according to their opinions rather than the research objectives. The layout of a survey can influence responses in various ways. For example the wording of the question may affect how respondents respond to the question and interpret it in the same way (reliable) or whether the question is relevant to the subject you are interested in (valid), and if they are able to accurately answer (credible).

Respondents may also experience survey fatigue or a lack of engagement with the questions that are asked which reduces the chances of them offering honest answers. Lack of incentives or compensation could also deter respondents from filling out the questionnaire.

Online questionnaires can also pose a challenge for certain experimental designs such as studies of reaction time or position. It is challenging to control and measure the same variables across people due to the variations in browser settings operating systems, browser settings, and screen sizes.

The bottom line is that Web-based surveys can only be accessed by people who have keyboards and Internet literate. This excludes a large segment of the population. It is also difficult to Web researchers debriefing participants after the experiment window has ended.