Survey Design SKILL.md

This is the exact file agents read, including the credit header and licence notice. Save it as survey-design/SKILL.md.

---
name: survey-design
description: "Design surveys that produce trustworthy data: objectives tied to decisions, population and sample size with margins of error, neutral single-idea questions, answer formats and scales, ordering and skip logic, pilots, invitations and reminders, cleaning, segment analysis and reports with limitations. Use whenever someone needs to create a survey or questionnaire, run customer or employee feedback surveys, prioritise features with survey data, calculate sample size or fix leading survey questions."
license: CC-BY-4.0
metadata:
  author: Afeez Adebayo
  author_url: https://afeezadebayo.com
  author_linkedin: https://www.linkedin.com/in/afeez-adebayo/
  author_x: https://x.com/za_icon
  source: https://afeezadebayo.com/skills-md/productivity/research-learning/survey-design/
  category: productivity
  subcategory: research-learning
  version: "1.0.0"
  created: "2026-10-10"
  updated: "2026-10-14"
---

# Survey Design

Surveys answer "how many" and "how common" questions that interviews cannot: what share of customers want a feature, how satisfaction varies by segment, which problems are most frequent. They are easy to send and easy to get wrong. Leading or double-barrelled questions, unbalanced scales, self-selected samples, long questionnaires that people abandon and results reported without margins of error produce numbers that look precise but mislead.

This skill designs surveys that produce trustworthy data: defining objectives, choosing who to survey and how many, writing clear unbiased questions, choosing answer formats and scales, ordering and testing the questionnaire, maximising responses, analysing results with appropriate caution and reporting findings with sample sizes and limitations.

## Operating principles
1. **Objectives decide every question.** If a question does not inform a decision, cut it.
2. **Right people, enough of them.** Who answers matters more than how many.
3. **Neutral, simple wording.** One idea per question, no leading language.
4. **Short wins.** Five to ten minutes keeps completion and quality high.
5. **Test before sending.** Pilot with a few people to catch confusing questions.
6. **Report with limits.** Sample size, who responded and margin of error.

## Question writing rules
| Problem | Bad | Better |
|---|---|---|
| Leading | "How much do you love our new dashboard?" | "How satisfied are you with the new dashboard?" |
| Double-barrelled | "Is our product fast and easy to use?" | Two questions: speed, ease of use |
| Vague time | "Do you often use reports?" | "In the last 30 days, how many times did you use reports?" |
| Jargon | "How useful is our NRR dashboard?" | Use plain terms or explain |
| Hypothetical | "Would you pay for feature X?" | Ask about current spend and priority of problems |
| Missing options | Choices that do not cover all answers | Add "other" and "not applicable" |

## Answer formats
| Format | Use for | Notes |
|---|---|---|
| Single choice | One answer from a list | Make options mutually exclusive |
| Multiple choice | Several answers | Limit options; randomise order where sensible |
| Rating scale (5 or 7 points) | Satisfaction, agreement, likelihood | Balanced, labelled endpoints |
| Ranking | Priorities | Keep to 5 to 7 items |
| Numeric | Counts, amounts | Give units and ranges |
| Open text | Reasons, ideas | Use sparingly; one or two per survey |
| Net Promoter Score | Likelihood to recommend, 0 to 10 | Add a follow-up "why" question |

## Sample size rough guide
For a simple percentage from a random sample, the margin of error at 95% confidence is roughly 1 divided by the square root of the sample size: about ±10% with 100 responses, ±7% with 200, ±5% with 400 and ±3% with 1,000. Self-selected samples can be biased regardless of size, so who responds matters as much as how many.

## When to use, and when not to
Use it for customer satisfaction, feature prioritisation, market sizing inputs, employee engagement, event feedback and validating patterns found in interviews.

Don't use surveys to understand why people behave in complex ways (interviews do that better) or when you cannot reach a representative group.

## Inputs
- Objectives and decisions the survey informs.
- Target population and how to reach them.
- Hypotheses from interviews or data.
- Tool (Google Forms, Typeform, SurveyMonkey or similar).
- Timeline and incentive budget.

## Workflow

### Step 1. Write objectives
Three to five objectives, each linked to a decision.

### Step 2. Define the population and sample
Who exactly, how they will be reached, target number of responses, segments needed for comparison.

### Step 3. Draft questions per objective
Use the writing rules; one idea per question; plain language.

### Step 4. Choose formats and scales
Consistent scales throughout; include "not applicable" where needed.

### Step 5. Order the survey
Easy, relevant questions first; sensitive and demographic questions last; group by topic; use logic to skip irrelevant questions.

### Step 6. Pilot
Five to ten people complete it while thinking aloud; fix confusion, length and errors.

### Step 7. Send and remind
Clear invitation with purpose, time required and privacy; one or two reminders; incentives if appropriate.

### Step 8. Clean and analyse
Remove duplicates, speeders and test responses; analyse by segment; report sample sizes and margins.

### Step 9. Report
Key findings with numbers, charts, comparisons, open-text themes and limitations.

## Survey invitation template
```
Subject: 5 minutes to help shape <product or service>
We're asking <who> about <topic> so we can <decision or improvement>.
It takes about 5 minutes and closes on <date>.
<Incentive, if any>. Answers are <anonymous / confidential> and used only for <purpose>.
<Link>
```

## Analysis checklist
- [ ] Response rate and who responded compared with the target population.
- [ ] Results by key segments, with sample sizes shown.
- [ ] Margins of error for main percentages.
- [ ] Open-text responses coded into themes with counts.
- [ ] Differences tested or described cautiously when samples are small.

## Scale design tips
- Use the same direction for all scales (for example 1 = very dissatisfied, 5 = very satisfied).
- Label every point or at least both ends and the midpoint.
- Keep a neutral midpoint unless you deliberately want to force a choice.
- Avoid mixing 5-point and 10-point scales in one survey.

## Coding open-text answers
Read a sample of 50 to 100 answers, create a short list of themes, then tag every answer with one or more themes and count them. Keep a few verbatim quotes per theme. AI can draft themes and tags quickly; check a sample of its tagging by hand before reporting counts.

## Avoiding survey fatigue
Survey each customer group no more than a few times a year, coordinate across teams so people are not asked the same questions twice and share what changed because of their answers, which raises future response rates.

Report response rates every time.

## Edge cases and judgement calls
| Situation | What to do |
|---|---|
| Low response rate | Check who answered; weight carefully or report as indicative |
| Small segments | Combine or report as directional only |
| Sensitive topics | Anonymous surveys; careful wording; data protection |
| Customers already over-surveyed | Shorter surveys; fewer sends; use existing data |
| Results conflict with interviews | Look for sampling or wording issues; investigate further |
| International audiences | Translate and back-translate; adapt examples |

## Handoffs to related skills
- Planning: `research-planning`
- Interviews to explore first: `expert-user-interviews`
- Synthesis: `research-synthesis`
- Presenting results: the Communication and Presentations `data-storytelling` skill
- Customer feedback programmes: the CRM and Lifecycle skills

## Worked example
A product team planned a 40-question survey asking customers which of 15 features they wanted. The redesign set three objectives, cut the survey to 12 questions, replaced "would you use" questions with questions about how often customers faced each problem in the last month and how they solved it, added a ranking of five problems and piloted with eight customers, which removed two confusing questions. With 620 responses (about ±4% margin), the team found two problems far more frequent than the rest and prioritised them, confirming the result with follow-up interviews.

## Common mistakes
- Long surveys with no clear objectives.
- Leading or double-barrelled questions.
- Asking about hypothetical behaviour.
- Reporting percentages without sample sizes.
- Treating self-selected responses as representative.

## Output format
```
1. Objectives and decisions
2. Population, sample and margin of error
3. Questionnaire with formats and logic
4. Pilot findings and changes
5. Invitation and reminder plan
6. Analysis plan and report outline
```

## Self-check before you answer
- Does every question map to an objective?
- Are questions neutral, single-idea and about real behaviour?
- Is the sample appropriate, with margin of error stated?
- Was the survey piloted before sending?

---
Created by Afeez Adebayo, Marketing, SEO & Growth Expert
Website: https://afeezadebayo.com
LinkedIn: https://www.linkedin.com/in/afeez-adebayo/
X: https://x.com/za_icon
Source: https://afeezadebayo.com/skills-md/productivity/research-learning/survey-design/

Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). You may copy, share and adapt this skill, including for commercial use, as long as you credit Afeez Adebayo and link to the source URL above. Keep this notice in any copy or adapted version.