Why should a researcher be wary of results of an analysis using a convenience sample?

Sampling > Convenience Sampling (Accidental Sampling)

Watch the video for an overview of convenience sampling:

Can’t see the video? Click here.

Why should a researcher be wary of results of an analysis using a convenience sample?

Convenience sampling (also called accidental sampling or grab sampling) is where you include people who are easy to reach. For example, you could survey people from:


  • Your workplace,
  • Your school,
  • A club you belong to,
  • The local mall.

Convenience sampling is a type of non-probability sampling, which doesn’t include random selection of participants. The opposite is probability sampling, where participants are randomly selected, and each has an equal chance of being chosen.

Why Use Convenience Sampling?

Although convenience sampling is, like the name suggests—convenient—it runs a high risk that your sample will not represent the population. However, sometimes a convenience sample is the only way you can drum up participants. According to Barbara Sommer at UC Davis, it could be “…a matter of taking what you can get”.

Convenience sampling does have its uses, especially when you need to conduct a study quickly or you are on a shoestring budget. It is also one of the only methods you can use when you can’t get a list of all the members of a population. For example, let’s say you were conducting a survey for a company who wanted to know what Walmart employees think of their wages. It’s unlikely you’ll be able to get a list of employees, so you may have to resort to standing outside of Walmart and grabbing whichever employees come out of the door (hence the name “grab sampling”).


Advantages of Convenience Sampling

  • It’s relatively easy to get a sample.
  • It’s inexpensive, compared to other methods.
  • Participants are readily available.

Disadvantages of Convenience Sampling

The method cuts out a large part of the population. As a result, this leads to several issues, including:


  • An inability to generalize the results of the survey to the population as a whole.
  • The possibility of under- or over-representation of the population.
  • Biased results, due to the reasons why some people choose to take part and some do not.

How to Analyze a Convenience Sample

Results from these samples are easy to analyze but hard to replicate. While you can use any analysis method you like, you won’t be able to generalize your results to the larger population.

Perhaps the biggest problem with convenience sampling is dependence. Dependent means that the sample items are all connected to each other in some way. This dependency interferes with statistical analysis. Most hypothesis tests (e.g. the t-test or chi-square test) and statistics (e.g. the standard error of measurement), have an underlying assumption of random selection, which you do not have. Perhaps most problematic is the fact that p-values produced for convenience samples can be very misleading.

Recommendations for analysis

The biggest recommendation is simple: If possible, use probability sampling (Berk & Freedman, 2003). Other recommendations:

  • Take multiple samples over the course of your study. If you do this, you may be able to model the selection process, producing more reliable results.
  • Don’t use post-hoc tests as a tool to adjust your results in an attempt to deal with dependent data.
  • Repeat your study again, to see if your results are truly replicable (Freedman, 1991; Berk, 1991; Ehrenberg and Bound, 1993).
  • For larger samples, use cross validation to model one half of the data. You can then compare the results with the second half of the data to see if they match.
  • Don’t meta analyze convenience samples. Meta-analysis combines the findings from existing research into one, comprehensive thesis. A meta-analysis can uncover trends or themes that weren’t apparent in individual pieces of research. If you’re using biased data from convenience samples, then any “trends” you uncover are going to be suspect. Summarize results instead.

References:

Berk R. A. (1991) “Toward a Methodology for Mere Mortals,” in P. V. Marsden (ed.), Sociological Methodology, Volume 21, Washington, D. C.: The American Sociological Association.
Berk,R. and Freedman,D. (2003) Statistical Assumptions as Empirical Commitments. In TG Blomberg and S Cohen, eds. Law, Punishment, and Social Control: Essays in Honor of Sheldon Messinger. Aldine de Gruyter, New York. pp. 235-54.
Ehrenberg A. S. C. and Bound J. A. (1993) “Predictability and Prediction,” Journal of the Royal Statistical Society, Series A, 156 Part 2: 167–206.
Freedman D. (1991) “Statistical Models and Shoe Leather,” in P. V. Marsden (ed.). Sociological Methodology, Volume 21, Washington, D. C.: The American Sociological Association.
Sommer, B. (n.d.). Types of samples. Retrieved September 13, 2017 from: http://psc.dss.ucdavis.edu/faculty_sites//sommerb/sommerdemo/sampling/types.htm

---------------------------------------------------------------------------

Why should a researcher be wary of results of an analysis using a convenience sample?

Need help with a homework or test question? With Chegg Study, you can get step-by-step solutions to your questions from an expert in the field. Your first 30 minutes with a Chegg tutor is free!

Comments? Need to post a correction? Please Contact Us.


What is the problem with using convenience samples for research?

Because the generalizability of convenience samples is unclear, the estimates derived from convenience samples are often biased (i.e., sample estimates are not reflective of true effects among the target population because the sample poorly represents the target population).

What is a convenience sample and why is it bad?

A convenience sample is one that is drawn from a source that is conveniently accessible to us. This sample, however, may not be representative of the population at large.

Why would a survey of a convenience sample not be trustworthy?

There is always a chance that the randomly selected population may not accurately represent the population of interest, thus increasing the chances of bias.

What is the largest problem with samples of convenience?

Disadvantages of Convenience Sampling It is prone to several forms of bias, including selection bias, researcher bias, and sampling bias. All of these biases can distort your research outcomes significantly. Convenience sampling often leads to a high level of sampling error in a systematic investigation.