Organizational Research: Determining Appropriate Sample Size in Survey Research 45
of error exceeds the acceptable margin of error;
i.e., the probability that differences revealed by
statistical analyses really do not exist; also known as
Type I error. Another type of error will not be
addressed further here, namely, Type II error, also
known as beta error. Type II error occurs when
statistical procedures result in a judgment of no
significant differences when these differences do
indeed exist.
Alpha Level. The alpha level used in
determining sample size in most educational
research studies is either .05 or .01 (Ary, Jacobs,
& Razavieh, 1996). In Cochran’s formula, the
alpha level is incorporated into the formula by
utilizing the t-value for the alpha level selected
(e.g., t-value for alpha level of .05 is 1.96 for
sample sizes above 120). Researchers should
ensure they use the correct t- value when their
research involves smaller populations, e.g., t-value
for alpha of .05 and a population of 60 is 2.00. In
general, an alpha level of .05 is acceptable for most
research. An alpha level of .10 or lower may be
used if the researcher is more interested in
identifying marginal relationships, differences or
other statistical phenomena as a precursor to
further studies. An alpha level of .01 may be used
in those cases where decisions based on the
research are critical and errors may cause
substantial financial or personal harm, e.g., major
programmatic changes.
Acceptable Margin of Error. The general rule
relative to acceptable margins of error in
educational and social research is as follows: For
categorical data, 5% margin of error is acceptable,
and, for continuous data, 3% margin of error is
acceptable (Krejcie & Morgan, 1970). For
example, a 3% margin of error would result in the
researcher being confident that the true mean of a
seven point scale is within ±.21 (.03 times seven
points on the scale) of the mean calculated from the
research sample. For a dichotomous variable, a
5% margin of error would result in the researcher
being confident that the proportion of respondents
who were male was within ±5% of the proportion
calculated from the research sample. Researchers
may increase these values when a higher margin of
error is acceptable or may decrease these values
when a higher degree of precision is needed.
Variance EstimationVariance Estimation
A critical component of sample size formulas is the
estimation of variance in the primary variables of
interest in the study. The researcher does not have
direct control over variance and must incorporate
variance estimates into research design. Cochran
(1977) listed four ways of estimating population
variances for sample size determinations: (1) take
the sample in two steps, and use the results of the
first step to determine how many additional
responses are needed to attain an appropriate
sample size based on the variance observed in the
first step data; (2) use pilot study results; (3) use
data from previous studies of the same or a similar
population; or (4) estimate or guess the structure of
the population assisted by some logical
mathematical results. The first three ways are
logical and produce valid estimates of variance;
therefore, they do not need to be discussed further.
However, in many educational and social research
studies, it is not feasible to use any of the first three
ways and the researcher must estimate variance
using the fourth method.
A researcher typically needs to estimate the
variance of scaled and categorical variables. To
estimate the variance of a scaled variable, one must
determine the inclusive range of the scale, and then
divide by the number of standard deviations that
would include all possible values in the range, and
then square this number. For example, if a
researcher used a seven-point scale and given that
six standard deviations (three to each side of the
mean) would capture 98% of all responses, the
calculations would be as follows:
7 (number of points on the scale)
S = ---------------------------------------------
6 (number of standard deviations)
When estimating the variance of a dichotomous
(proportional) variable such as gender, Krejcie and
Morgan (1970) recommended that researchers
should use .50 as an estimate of the population
proportion. This proportion will result in the
maximization of variance, which will also produce
the maximum sample size. This proportion can be
used to estimate variance in the population. For
example, squaring .50 will result in a population