Does potential non-anonymity affect responses to potentially controversial questions?: Independent Research
Nov 29, 2018
3 minute read

I’ve just completed a bit of independent research for an AP Statistics project, feel free to take a look at the poster!

Alternatively, here’s the text:

Does potential non-anonymity affect responses to potentially controversial questions?

Introduction

This poster proposes and tests a potential issue with surveying techniques: does the presence of an email field, optional or not, affect the results of a survey? I propose that an email field will decrease potentially controversial responses, even if it is not filled. To accomplish this, I have constructed a program (named “surveyor”) which runs on a server and hosts a single web page: a survey. surveyor has a bank of two surveys, one where the first question is asking for an email address and one where that question isn’t present, and displays a random one to the user. It stores the displayed survey in a browser cookie, so that refreshing will give the user the same survey. The survey I designed poses as a demographic survey for the GSRM (gender, sex, and romantic minority) community, and as such contains many demographic questions. I will be focusing on one, where respondents rate their sexual orientation on the Kinsey scale. I have phrased this question as attraction to men/women as opposed to same/opposite gender, however I have standardized the results to the latter based on gender given by respondents. Unfortunately, this means I must filter out non-binary and indeterminate gender respondents, as including them would make data processing much harder.

Data Collection

Data was collected by distributing a link to the survey to many online GSRM communities: various chat rooms which I am a member of, Mochi, reddit.com/r/lgbt, reddit.com/r/LesbianActually, reddit.com/r/bisexual, and reddit.com/r/LGBTeens. I collected 40 responses, though some were ignored for above reasons.

Potential Error

Given that the survey was only distributed to GSRM communities, respondents may be more comfortable responding with their true sexual orientation. This limited population also resulted in a smaller-than-expected sample size of $n=40$. For the first few respondents, the software did not randomize the given survey, so they all received the non-email survey. This was fixed in the code promptly.

Statistics

The graphs are present in the PDF linked at the top.

The above histograms show the Kinsey scale from 1 (perfectly heterosexual) to 7 (perfectly homosexual) for the two survey types (no email and email respectively). Survey 0 has $\bar{x}=5.78$ and $s=1.53$. Survey 1 has $\bar{x}=5.48$ and $s=1.83$. The x-axis is the Kinsey scale and the y-axis is the proportion of respondents answering the respective Kinsey scale value.

Conclusion

There is a small, yet present, difference between $\bar{x}$ of each survey, which indicates that further research, with the above potential error corrected, is necessary to conclusively determine if presence of an email field has an effect.