Sample Size and Margin of Error in Employee Surveys

Sample size is the number of responses a survey result is based on; margin of error shows how far that result could deviate by chance from the true value you would get if all employees responded. In employee surveys, the margin of error at 95% confidence is calculated with the formula z × √(p × (1 − p) / n) × √((N − n) / (N − 1)), where N is the number of employees, n the number of responses and p the favorable score. For example, when 500 people respond in a 1,000-person organization, the margin of error is about ±3 points; in a 50-person unit with 25 responses it rises to ±14 points. In this article we explain the formula in plain language, share tables calculated by organization size and response rate, and show what to watch for when interpreting results.

Census or Sample?

Election polls and market research are carried out with a few hundred or a few thousand people selected to represent a large population. This is called a sample. Employee surveys usually work differently: the survey is sent to every employee in the organization. A survey that everyone answers is called a census, and a census has no random sampling error.

In practice, no employee survey gets a response from everyone. A survey completed with a 70% response rate does not include the views of 30% of employees. So a survey sent to everyone effectively becomes a sample because of the response rate, and the result carries a margin of error. That is why saying "We asked all employees, so the result is exact" is not correct.

There are two more important differences:

• The sample is not randomly selected: Respondents decide for themselves whether to take part. This creates a risk the margin of error does not capture (we cover it below under non-response bias).
• The population is small and known: In an election poll the population is in the millions; in an employee survey it is 80, 600 or 4,000 people. That is why a finite population correction is added to the formula.

What Are Margin of Error and Confidence Level?

Margin of error is the range of uncertainty around a result. If the favorable score on the engagement index is 62% and the margin of error is ±4 points, you can say that had all employees responded, the score would most likely have been between 58% and 66%.

Confidence level is how likely that "most likely" is. A 95% confidence level means that if the same survey were repeated many times under the same conditions, about 95% of the calculated ranges would contain the true value. Survey research uses 95% as the standard. The higher the confidence level, the wider the range: with 99% confidence you get a wider margin of error from the same data.

Three things determine the margin of error:

1. Number of responses (n): The more responses, the narrower the margin; but because of the square-root relationship, halving the margin of error requires roughly four times as many responses.
2. The proportion itself (p): The margin is widest when the proportion is 50%. It is narrower at extreme proportions such as 90% or 10%.
3. Population size (N) and coverage: When a large share of employees respond, the finite population correction narrows the margin noticeably.

The Margin of Error Formula

For a proportion (for example an item's favorable score), the margin of error is calculated as follows:

e = z × √(p × (1 − p) / n) × √((N − n) / (N − 1))

• e: The margin of error. The result is a decimal; multiply by 100 to convert it to points (0.043 = ±4.3 points).
• z: The coefficient for the confidence level: 1.645 for 90%, 1.96 for 95%, 2.576 for 99%.
• p: The favorable score as a decimal (for example 0.70 for 70%). If you don't know it yet, use 0.50, which gives the widest margin.
• n: The number of respondents.
• N: The total number of employees invited to the survey.
• √((N − n) / (N − 1)): The finite population correction. When a large share of employees respond, it falls clearly below 1 and shrinks the margin of error. When everyone responds (n = N), it becomes zero.

Example: In an organization of 1,000 employees, 300 people responded and an item's favorable score is 70%.

• Standard error: √(0.70 × 0.30 / 300) ≈ 0.0265
• Finite population correction: √(700 / 999) ≈ 0.837
• Margin of error: 1.96 × 0.0265 × 0.837 ≈ 0.043, i.e. ±4.3 points

Without the correction, the margin of error would be ±5.2 points. Had 700 people responded in the same organization, it would drop to ±1.9 points.

Margin of Error by Organization Size and Response Rate

The table below shows margins of error calculated at 95% confidence with the most conservative assumption, p = 0.50. The number in brackets is the number of responses.

Values are in points and calculated with the finite population correction. The further the favorable score is from 50%, the slightly narrower the margin; for example, at 80% it is about 80% of the values in the table.

Three practical conclusions from the table:

• In large organizations, even a low response rate gives a narrow margin. In a 5,000-person organization, a 30% response rate gives a margin of ±2.1 points. In large organizations the real risk is not the margin of error but who did not respond.
• Small units need a high response rate. In a 100-person unit, a 30% response rate gives a ±15-point range, which makes the result almost impossible to interpret.
• The gain from higher participation is larger in small groups. In a 250-person unit, raising participation from 50% to 70% cuts the margin from ±6.2 to ±4.1 points.

How Many Responses Do You Need for a Given Margin of Error?

You can also ask the question the other way round: "If I want a ±5-point margin of error, how many responses do I need?" First the number needed for an infinite population is calculated, then the finite population correction gives the number needed for your organization:

n₀ = z² × p × (1 − p) / e²

n = n₀ / (1 + (n₀ − 1) / N)

At 95% confidence and p = 0.50, n₀ is about 385 for ±5 points and about 1,068 for ±3 points. As your organization grows, the required number approaches these values but never exceeds them.

Calculated at 95% confidence and p = 0.50; response counts are rounded up.

This table explains why a 7% response rate can look "statistically sufficient" in a large organization. But that only holds for the organization as a whole. When you break results down by unit, location and collar type, each group becomes its own small population and each group needs enough responses. A 7% response rate also carries a very high representativeness risk.

Why Do Small Teams Have Wide Margins?

In small groups, each individual has a large effect on the result. In a 15-person team where 12 people respond, one person changing their answer from "Neutral" to "Agree" moves the favorable score by about 8 points. In this team, a 75% result has a margin of error of around ±11 points, even with the finite population correction.

This should not be confused with the 10-person reporting threshold organizations apply. That threshold exists to prevent anyone from guessing whose answers are whose, i.e. to protect anonymity, not to ensure statistical precision. When 10 people respond in a 20-person team, the result can be reported, but its margin of error at 95% confidence is close to ±22 points. We explain the anonymity rules in anonymity in employee surveys.

When interpreting small-team results:

• Read small differences (5–10 points) between a team score and the organization average as "no difference".
• Focus on category or index scores rather than individual items; the average of several items fluctuates less.
• Avoid ranking teams against each other. In small teams, rankings depend largely on chance.
• Treat open-ended comments and team feedback meetings as being as important as the scores.

We explain how to define groups and breakdowns in segmentation in employee surveys.

Response Count Is Not Representativeness

The margin of error formula assumes that respondents were randomly selected from among employees. In employee surveys, however, participation is voluntary and respondents may differ from non-respondents. This is called non-response bias, and the margin of error does not account for it.

For example, in a 2,000-person organization with a 70% response rate, the margin of error is only ±1.4 points. But if most of the 600 non-respondents are shift workers at the factory and this group is less engaged than office employees, the organization-wide result looks better than it really is. The narrow margin here gives a misleading sense of precision.

To limit non-response bias:

• Track participation not only for the organization as a whole but also by unit, location, collar type and tenure.
• Identify groups with low participation while the survey is still open and support them with reminders, kiosks or paper forms.
• Compare the distribution of respondents with the distribution of employees; if there is a clear gap, interpret the organization result group by group.
• Show the participation breakdown clearly in the results report.

We explain what level of participation counts as sufficient in employee survey response rate, and how to raise it in our guide how to increase employee survey response rate. We cover other types of bias in survey response bias.

Practical Guidance

1. Send it to everyone. In employee surveys, inviting all employees rather than selecting a sample both narrows the margin of error and gives every employee a voice. Sampling is worth considering only for very frequent short pulse surveys or to reduce survey fatigue in very large organizations.
2. Decide the smallest group you will report on up front. Set the participation target for that group. A 30% rate may look sufficient for the organization as a whole, but it is far too low for 60-person units.
3. Show the margin of error in reports. At least for small groups, show the number of responses next to the result or display the confidence interval. This stops managers from overreacting to small differences.
4. Test year-over-year differences separately. Whether the difference between two measurements is meaningful depends on the margins of both. A significance test for the difference between two proportions is used for this. For this comparison we recommend a more cautious approach without the finite population correction; details are in comparing engagement survey results year over year.
5. Assess precision together with representativeness. A narrow margin of error is meaningful when respondents represent the organization well.

Common Mistakes

• Assuming a census: Thinking the result is exact because the survey was sent to everyone. If there are non-respondents, the result carries a margin of error.
• Applying the organization-wide margin to units: A margin of ±2 points for the organization can exceed ±10 points in an 80-person unit.
• Mistaking the reporting threshold for an adequate sample: The 10-person threshold is for anonymity. A group with 10 responses can be reported, but its result should be read with a very wide range.
• Turning small differences into rankings: Ranking units as "best" and "worst" on differences within the margin of error leads to unfair comparisons between managers.
• Mistaking a high response count for representativeness: Thousands of responses do not fix the fact that a particular group is systematically missing.
• Mixing up the proportion formula and mean scores: The formula in this article is for proportions (for example the favorable score). For mean scores on a 1–5 scale, the margin of error is calculated using the standard deviation of the scores.

Knowing the margin of error lets you interpret survey results more cautiously but with more confidence: you can tell which differences are real and which are chance, and direct managers' energy to the areas that truly matter. We explain how consistent the measurement itself is in survey reliability and validity.

Frequently asked questions