How much confidence shall be placed in claims exerted by "Survey Reports and Statistical Inferences?
Before you launch a product or promote a service, the first question which comes to mind will be, “how the target audience / customers” will respond to the idea? The quickest way to find the answer is invariably by conducting a survey of the sample population from target segment. While, Survey Research is one of the easiest modes of conducting research, either for marketing or for study of human psychology, but caution must be exercised, when using the findings or relying on the published data, based on survey reports ..........
The level of reliability and confidence in the findings of a survey research depends largely on three critical factors:
1. Survey Design
2. Mode of Survey
3. Data Analysis Methodology
Considerable focus is placed on the design of survey instruments (Questionnaires, interviews, observations etcetera) and vocabulary, wording / phrases used, when drafting the questions. Despite a good design, training of the surveyor is equally important for consistency in implementing the plan.
The quality and reliability of responses further depends on the mode of survey, which can be by one or more of the following means:
a) Survey by Posting the questionnaires
b) Telephone Interviews
c) Face to Face Interviews
d) Web based Survey
e) Customised in accordance with the context of the research
During the data analysis stage, there is very little scope for any improvement in the quality of responses. As Consultants or Analysts will rely only on the data received, and conduct the review within the context of the Survey Design. Thus to enhance the reliability of the survey and confidence level in the findings, the users of the report shall be made aware of the commonly found “Errors in Survey Exercises”.
Errors in Survey Exercises
Leaving aside administrative and human errors like manipulation and or subjectivity of the data, there are three well recognized errors in this research methodology.
1. Sampling Error: This is a random error and described as non-directional or non-systematic error, because measurements can be either too high or too low. Sampling error occurs because a sample from the whole population is surveyed instead of full population. There are statistical methods to compute errors due to sampling when simple random sample is taken. Also to minimize the error and thus maximize the confidence level, different methods of sampling are adopted, depending upon the context of the survey exercise.
2. Non-response Error: This may not be a serious problem but from a sample if some didn’t respond, their views could be different from who responded. At times, to get the minimum sample size, additional samples are drawn from the population to meet the requirement of survey design / sample size, this will create deviation from the designed parameter, which may or may not have any significant impact on the confidence level of the findings.
3. Response Errors: This is the most significant type or error among all the errors in survey exercises. The respondents may not give proper answer or exaggerate their response, for example in response to the age for females, income for males, and gender discrimination at work place etcetera. Biased questions may also encourage such responses. The response error is also dependent on the surveyor (Trained or untrained, male or female), place, environmental influences, timings of the day, mood of the respondent and mode of survey. Web survey and email based survey can to a large extent, reduce the surveyor based errors. Also the respondents can choose to reply at their convenient time and place, which are more likely to be a rational response.
As the internet based survey applications are getting more popular and cost effective, survey based research is becoming more affordable. However the cruciality of survey design is still a matter of paramount importance. Thus the users of the survey reports are advised to be “sceptical” about the findings and statistics published based on surveys. The measures of reliability like error and confidence level shall always be accompanied with the reports.
Visitors to the TMC website, www.tiberiasmc.com are offered an opportunity to participate in the monthly poll of wide ranging subjects, and see the statistics on the spot. The results can also be reviewed at weekly intervals. At the end of the month and subject to sufficient responses to the “Poll of the Month”, a brief analysis will be published.
So do participate in this month’s poll and gain insight on how your opinion matches with others, like yourself!
Thank You!
