Agenda - Council Work Session - 03/11/2014

Home

elicit more candid responses than surveys administered by phone'. Whether statistically more positive or simply somewhat more positive, the cumulative effect of phone response (to questions about service quality or service satisfaction) is to give local government officials an inaccurate measure of their effectiveness. This does more for the ego than it does for management. When officials go to the trouble and expense to hear from their residents, they don't want to create the psychological pressure that diners feel at a restaurant when the waitperson asks, "So how is the meal ?" The data gathered in response to that table question are notoriously inaccurate and of virtually no assistance to the waitperson or the manager and appear only to be asked to assure the diner that someone seems to care. Relevant to citizen surveys, one vaunted survey researcher concludes, "...people are invested in their neighborhood or community and are predisposed to put this environment in a positive light when interacting with an outsider (the interviewer)." (p.231) Genuine resident perspective about local government services comes reliably when residents have the time and privacy to reflect on citizen survey questions and then to respond anonymously without worry that an interviewer, however well trained at neutrality, may take offense. 6. Misguided mixing As self deceiving as service evaluations by phone can be, mixing results from phone with other modes is particularly misguided. Although it seems that giving respondents several ways to respond (phone, mail, e -mail, in- person) increases respondent's convenience, results will not be valid and response rates aren't improved. Overall inflated evaluations (from phone or in- person) will be blended with more honest evaluations (by mail or Web) and the stew that is created is not the meal paid for. Respondent convenience is an important characteristic of survey research but convenience should never trump the validity of findings. If convenience were all that mattered in survey research, an ad could be placed in the paper or survey forms offered at the library, and we know that when these straw polls produce useful results, it is by accident. It is deceiving enough to mix evaluations from two data collection sources but some market research firms collect as much data as they can by mail, first, and then follow with phone calls to those who have not returned the mailed questionnaire. By this approach, residents with unlisted numbers cannot be surveyed by phone because the only way to get the phone number for a particular address where a survey has been mailed is to run the address through what is called a reverse directory. Typically only 40% to 60% of addresses these days have listed phone numbers, but the rest do not. By missing phone interviews with residents whose numbers are unlisted, double harm is done. First, responses of those interviewed by 2 Dillman, D A Mail and Internet Surveys: The Tailored Design Method. Wiley & Sons, Inc. New York. 2000. © National Research Center, Inc. 2955 Valmont Rd, Suite 300, Boulder, CO 80301 www.n- r -c.com 303 - 444 -7863