Choose a method that can answer the question

An opinion survey cannot measure every outcome

• The materials team asks, “Do you like our new label layout?” A positive answer may show preference, but it does not show whether the reader can sort the four items correctly. The method must match the outcome in the question.
• This original GAC033-aligned lesson compares an observation, a survey and a small experiment. All are useful for some questions. None is automatically the best method for every topic, and an elaborate method cannot repair a vague research question.

Compare what each method tells you

• An observation can record what occurs in a setting, but recording errors alone may not reveal why they happen. A survey can ask about experience or preference, but answers may differ from behaviour. Avoid reporting either as evidence of an outcome it did not measure.
• An experiment deliberately changes a feature and compares an outcome under planned conditions. For the label project, the team proposes changing layout while keeping words, item choices, time and instructions the same. Poorly controlled differences would weaken the interpretation.

Plan a fair small comparison

• The proposal uses 24 willing peers: 12 see A and 12 see B, assigned by shuffled slips rather than the team's preference. Each completes the same four-item paper sorting task within two minutes, without coaching. The labels' wording remains the same; spacing and alignment differ.
• Score one completely correct sheet only when all four placements match the key fixed before the task. Record the number of complete successes in each group, not a changing mixture of partial marks and whole-sheet counts. No main study has been conducted yet.

{callout:example} Outcome rule: four correct placements = one success; any placement incorrect or blank = no complete success. A separate error-category note may aid interpretation, but it must not silently change the primary outcome. {/callout}

Describe the sample honestly

• A sample is the participating group. Volunteers from one class are not all students everywhere. Random assignment within that group can make the A/B comparison fairer, but it is different from randomly sampling the entire school.
• State who is invited, how many are planned and what happens if fewer agree. Do not recruit only confident readers for B and less confident readers for A. With a small group, random assignment still does not guarantee identical groups in every characteristic.

{callout:warn} Do not run a participant task before the teacher approves the procedure. Participation must be voluntary, separate from grades, and explained without pressure from teammates or teachers. {/callout}

Make an ethical procedure explicit

• Explain the task, purpose and use of anonymous totals before anyone agrees. Participants may stop or leave a sheet unsubmitted. Collect no names or identifying comments. Explain that after a sheet is submitted without an identifier, the team cannot reliably retrieve one person's data.
• Use ordinary printed materials, not a task that risks damage, embarrassment or disclosure. Keep records in the approved project location and agree access and disposal with the teacher. “No names” does not excuse careless handling of other identifying details.

Pilot the procedure, not your conclusion

• With approval, ask two people to check whether instructions are understandable and timing is workable. This pilot is a rehearsal of the method. If the team changes instructions afterward, keep those pilot sheets out of the main comparison.
• Write a protocol someone else could follow: recruitment, assignment, materials, instructions, clock, scoring, privacy and analysis. Mark a sentence that could lead two helpers to run the task differently and make it precise before any main collection.

{callout:key} Match the method to the question, define the outcome in advance, control relevant differences and make sample and ethical limits explicit. {/callout}