Does potential non-anonymity affect responses to potentially controversial questions?: Independent Research
Nov 29, 2018
3 minute read

I’ve just completed a bit of independent research for an AP Statistics project, feel free to take a look at the poster!

Alternatively, here’s the text:

# Does potential non-anonymity affect responses to potentially controversial questions?

## Introduction

This poster proposes and tests a potential issue with surveying techniques: does the presence of an email field, optional or not, affect the results of a survey? I propose that an email field will decrease potentially controversial responses, even if it is not filled. To accomplish this, I have constructed a program (named “surveyor”) which runs on a server and hosts a single web page: a survey. surveyor has a bank of two surveys, one where the first question is asking for an email address and one where that question isn’t present, and displays a random one to the user. It stores the displayed survey in a browser cookie, so that refreshing will give the user the same survey. The survey I designed poses as a demographic survey for the GSRM (gender, sex, and romantic minority) community, and as such contains many demographic questions. I will be focusing on one, where respondents rate their sexual orientation on the Kinsey scale. I have phrased this question as attraction to men/women as opposed to same/opposite gender, however I have standardized the results to the latter based on gender given by respondents. Unfortunately, this means I must filter out non-binary and indeterminate gender respondents, as including them would make data processing much harder.

## Data Collection

Data was collected by distributing a link to the survey to many online GSRM communities: various chat rooms which I am a member of, Mochi, reddit.com/r/lgbt, reddit.com/r/LesbianActually, reddit.com/r/bisexual, and reddit.com/r/LGBTeens. I collected 40 responses, though some were ignored for above reasons.

## Potential Error

Given that the survey was only distributed to GSRM communities, respondents may be more comfortable responding with their true sexual orientation. This limited population also resulted in a smaller-than-expected sample size of $n=40$. For the first few respondents, the software did not randomize the given survey, so they all received the non-email survey. This was fixed in the code promptly.

## Statistics

The graphs are present in the PDF linked at the top.

The above histograms show the Kinsey scale from 1 (perfectly heterosexual) to 7 (perfectly homosexual) for the two survey types (no email and email respectively). Survey 0 has $\bar{x}=5.78$ and $s=1.53$. Survey 1 has $\bar{x}=5.48$ and $s=1.83$. The x-axis is the Kinsey scale and the y-axis is the proportion of respondents answering the respective Kinsey scale value.

## Conclusion

There is a small, yet present, difference between $\bar{x}$ of each survey, which indicates that further research, with the above potential error corrected, is necessary to conclusively determine if presence of an email field has an effect.