Conjoint questionnaire planner
How many choices should you ask people to make?
Plan the length of your conjoint exercise, see what each person will evaluate, and compare the time needed for different approaches.
Start with your study
For conventional choice-based conjoint: every product is described by the full set of features. The default is 12 learning questions, 2 validation questions, and 1 practice question. All are editable planning assumptions.
Your questionnaire plan
What each person will answer
- 12 learning
- 2 validation
- 1 practice
- Conjoint exercise
- 7.3–12.0 min
- Whole survey
- 12.3–17.0 min
Within your 20-minute limit at the slower assumed pace.
Starting point to test: 12 learning questions
The 12-question starting point fits the entered time allowance. Test the design and predictions before choosing the final length.
Planning guidance only. Design quality, precision, and predictive performance remain unassessed.
A view of one choice question
Illustrative layout only. 18 feature entries per question; 216 across the learning questions.
3 products × 6 features, plus a None option. Display counts do not measure difficulty or statistical information.
Before you field the survey
- Test the actual questionnaire on a phone, including text size, scrolling, and comparisons between products. Device choice does not apply a time or quality multiplier.
Calculation details and your plan
Time: instructions and all practice + (learning + validation) × seconds per choice ÷ 60 + follow-up time. Add the rest of the survey for the whole-survey estimate.
Sample reference, when enabled: round up [appearance reference × maximum levels ÷ (learning questions × products)]. None, practice, validation, and follow-up responses do not add estimation appearances.
Suggestion: use the count closest to the situation’s starting point within its planning range that fits the slower time estimate and, if enabled, the sample appearance check. Ties favor the shorter exercise. These checks do not establish statistical adequacy.
15 choice questions. Estimated whole survey 12.3–17.0 minutes. Within the time limit at the slower assumed pace.
Compare questionnaire lengths
What changes if you ask more or fewer?
These calculations hold your products, features, validation, practice, and timing assumptions constant. Select a row to try that length.
| Learning questions | Total choices | Whole survey | Time limit |
|---|---|---|---|
| 11 | 10.7–14.0 min | Within | |
| 13 | 11.5–15.5 min | Within | |
| 15 | 12.3–17.0 min | Within | |
| 17 | 13.2–18.5 min | Within | |
| 19 | 14.0–20.0 min | Within | |
| 23 | 15.7–23.0 min | Over |
“Within” refers only to the slower assumed time. Longer exercises are not automatically more predictive. Each length needs a valid experimental design.
Compare a different number of products per question
More questions can still mean less information on each screen. Compare the display counts for another layout; they do not establish which design is better.
Current layout
12 learning questions × 3 products × 6 features
- Entries on each screen
- 18
- Across learning questions
- 216
Alternative layout
16 learning questions × 2 products × 6 features
- Entries on each screen
- 12
- Across learning questions
- 192
After switching, review your timing assumptions. The planner does not assume that fewer products makes a question faster.
Using your plan
Choose a length by testing what it helps you learn.
The planner calculates question counts, display counts, and time scenarios. Its suggested length is a starting point for a real survey design. It cannot tell whether your pricing decision or product comparison will be precise enough.
Discuss your conjoint designWhy start with 12 learning questions?
Twelve is an editable starting point within our 8–16 planning range for a typical study. The short-survey, individual-preference, and technical-choice presets use different ranges. These are planning conventions, not research-established cutoffs.
Studies have found reliable responses in longer exercises, but reliability does not prove that preferences stay unchanged or that predictions improve. The evidence supports comparing lengths against the decisions you need to make. Johnson & Orme; Li et al.
What should a pilot compare?
Compare valid designs at different lengths, such as 8, 12, and 16 learning questions. Randomly assigning people to different lengths gives different evidence from using only the first eight answers in a longer survey; cutting a design short can also remove necessary coverage.
Assess predictions on choices excluded from model fitting, whether the business conclusions change, completion rates, and the distribution of completion times. Reserve separate data for final validation if you use holdouts to tune the questionnaire or model.
Do faster answers mean people have stopped paying attention?
Not on their own. People can learn how to compare the products and respond faster. Examine prediction, task-position effects, and changes in choice behavior alongside timing. Neither speed nor a small number of missed holdouts should be an automatic respondent-quality verdict. Meißner et al.
Can I use this for a survey with many features?
You can compare display counts, but there is no universal acceptable number of feature entries. Test the wording and layout on the devices people will use. Review whether every feature is needed and whether shared levels or a different conjoint format would make the task easier. Jonker et al.
Adaptive, partial-profile, menu-based, and volumetric conjoint need a method-specific assessment. Their screens are not interchangeable with full-profile choice questions.
Can more questions reduce the sample I need?
More questions increase observations from the same people. More respondents add people from the population. These are different sources of information, especially for subgroups and individual preferences.
The optional appearance calculation is a limited main-effects check. Use the sample-size estimator for overall and group budgeting bases and recruitment allowances, then assess the actual design and intended analysis.
Research behind the planner
Question count is only part of the decision.
The studies below examine different products, audiences, and definitions of response quality. They inform the guidance; they do not validate the default timings or produce a universal “best” length. Sources and calculation policy reviewed September 29, 2026.
- Johnson & Orme (1996). How Many Questions Should You Ask in Choice-Based Conjoint Studies?
Twenty-one commercial studies, with 3–6 attributes and 8–20 tasks. Later answers remained reliable, but the measured importance of brand and price changed. The analysis primarily assessed internal consistency.
- Bansak et al. (2018). The Number of Choice Tasks and Survey Satisficing in Conjoint Experiments.
Political conjoint experiments found limited deterioration through 30 tasks in the settings studied. The authors did not identify an optimal task count or recommend 30 tasks for every study.
- Meißner, Musalem & Huber (2016). Eye Tracking Reveals Processes that Enable Conjoint Choices to Become Increasingly Efficient with Practice.
Three eye-tracking studies connect faster decisions with more focused attention and improved reliability. Faster responding alone is not evidence of fatigue.
- Li et al. (2022). The More You Ask, the Less You Get: When Additional Questions Hurt External Validity.
The conjoint analysis used 70 participants and 20 laptop choices. Prediction of a separate, differently structured choice peaked early and then declined. This is evidence to test prediction, not a universal stopping rule.
- Jonker et al. (2019). Attribute level overlap (and color coding) can reduce task complexity, improve choice consistency, and decrease the dropout rate in discrete choice experiments.
A randomized study of 3,320 respondents supports testing shared feature levels and visual simplification. Its observed improvements are not applied as multipliers in this calculator.
- Ekstromer (2024). Respondent Fatigue in Choice-Based Conjoint: When and How Does It Affect the Results?
Sawtooth conference proceedings, pp. 345–355. Across three levels of complexity, later-task models predicted less well, while including more tasks could still improve the combined model. This does not set a universal maximum.
- Orme (2019). Sample Size Issues for Conjoint Analysis.
Getting Started with Conjoint Analysis, chapter 7. Source for the optional 500/1,000 appearance references. These are budgeting rules of thumb, not power calculations.