Filling in one questionnaire usually needs only several minutes to ten more minutes time.This simple and shortcut questionnaire filling experience can easily bring people an illusion:The questionnaire design is also very simple,ie,several questions adding to some options.Actually not so,the aim of questionnaire inquisition is to gain cognition of consumer understand by already designed question.Because it lacks of direct communication with users,the correctness and reliability of consumer understand entirely lies on the questionnaire itself.Good questionnaire design can educe useful surveying result.So,that how to design good questionnaire is a basic skill for user researchers that must learn and master.By analysing a real questionnaire,this text shows mistakes which easily appears on the questionnaire design,and comes into helping user researchers to understand several important principles of questionnaire design.
The example that this text adopts is gained from a recently ongoing questionnaire inquisition about "human based" status of software interface on one website.This questionnaire has 10 questions altogether,and is divided into two parts,aiming separatedly at investigation of the two groups of ordinary software users and developers.The text only analyses and discusses qeustion by question the first part of the questionnaire,ie,aiming at partial question of ordinary users.
1.Use correct language
The title of this part of questionnaire is:Is the software you are using is "human based"?According to the illumination in investigation result analysing,the aim of this part of questionnaire is to understand "as a user,what requirement and feeling has for the interface.Regrettably,such clear and plain description in the questionnaire "as a computer user,does the software which can help you complete work really has a "human based" interface?"This sentence with specialty taste is replaced .The word "human based" appears altogether five times before the questionnaire question,but there are no place to provide explanation to tell interviewers what is "human based".The key factor of user research is to regard users as core.This is also applied in the questionnaire design.The expected interviewers of this questionnaire are ordinary users.They may undertake all kinds of careers,receive different levels of education,and have different familiarity degree with computer.For ordinary users,"human based" is a word whose meanings are not easy to understood.Though qustionnaire designers try to transfer the aim explaination of this questionnaire to interviewers,its effect is largely discounted because of using too many professional glossaries.So,when designing questionnaire,designers should ever and again remind themselves that interview object is not themselves.The words which they think clear and inerrant are unnecessarily clear in the eyes of the interviewers.Only considerately stand in the interviewers' angle to choose words can make interviewers understand correctly the explanation,question and correct meanings of the options in the questionnaire,accordingly we can expect them to provide believable data.
2.Transfer exact information
For an interviewer,their main understanding of the investigated content is from the questionnaire itself(Especially when the undisciplined investigators often omit speaking volumes for interviewers because that they feel no necessarity or anxious to finish the investigation).So,every sentence and word in a questionnaire should be carefully went behind,and make them transfer true and inerrant information to interviewers,and let them first look at the first question of this questionnaire:
Which factor of the software do you think is the most important?
A.Good interface exceeds excellent function
B.Excellent function exceeds good interface
C.When first using,think that interface is important,after using for a long time,feel that the function is more important
D.When first using,think that function is very important,after using for a long time,fells that a comfortable interface is more important
If only look at this problem,the natural reaction of interviewers is to pick out one that they think the most important from several provided aspects.However,the problem is,this question provides four candidated answers,but in the first eye it has two aspects:interface and function.It's not the four aspects that we expect.This option's disharmony with the question is easy for interviewers to feel mazed,even because of this they will give up finishing the whole questionnaire.The reason that arouse this phasic is actually that the designers don't correctly give expression to the information which interviewers expect to figure out.If we carefully conjecture the relation among four options,it's easy for us to realize that actually designers want to know that whether interviewers's estimation of the interface and function which is subordinate and which is important will change along with increasing using experience.This aim of designers' is difficult for interviewers to realize because it uses irrelevant quizing ways.
3.Dispel wrong hypothesis
Sometimes,it is necessary to adopt some hypothesis for interviewers.Then look at the second question design:
The habit of your operating a software(such as Word):
A.Use default setting,remember the position of a commend(menu item or Toolbar button) and evey time directly find this position to choose it.
B.Use user-defined setting,place the command on an obvious position.
It's hard to imagine the software using habit of a user's is to choose one from two.These two option describe such a prospect:a user may choose the menu item from the default position for ever,or retailor all the menu item.Obviously,our experience tells us that at many times our behaviour is different from the two described option.As for the common-use function,we are likely to tailor it and make it more effective when using.In ther other hand,many functions are not used by us very often.We know their default position,so we also can find them when required.This question design place interviewers in a dilemma condition of non-this or non-that,which brings them extra choice difficulty.Because this result uses wrong hypothesis,once get rid of this wrong hypothesis,the mess that interviewers face won't exsist again.
4.Define integrated aggregation
When designing the choice subject of questionnaire,you should try best to make all the options together to compose a corpora.Only by this,it can probide interviewers a integrated option.This requires questionnaire designers to have correct understand of the relation among all options.Let us look at the third question of the questionnaire.
Which style of the interface do you like?
A.Entirely custom-built,ie,be able to decide whether all the command (meunu item or toolbar button) appear or not and where it appears.
B.Partly custom-built,ie,be able to decide the position of the command,but don't hide them.
C.Can't be custom-built,ie,all the command should have its own fixed position,and unable to change.
Only the words in the first half paragragh of the three options,"entirely custom-built","partly custom-built","can't be custom-built" correctly forms an integrated corpora.However,does this corpora represent the correct corpora?Let us carefully research the second half explanation of every option.From these three explanation,it's not difficult for us find that the word "customization" contains two meanings in the options of the question:the displayed position and whether it displays.So,this two-layer customization will produce two combination,and it's not the three kind in present design.The negligence place of this question is on the term of partly customization.The first and last option both correctly explain the meanings of each first half paragragh.And for the option of partly customization,its explanation of the second half paragraph don't make this part cover all the area which removes other two options.Thereby,it lead to the complexion of similarly corpora but not similarly corpora.
5.Provide transparent standard
In questionnaire investigation,a kind of question which is usually asked is frequency.For example,how many hours do you go on internet every day,how long has you changed a mobile,and so on.When designing this kind of questions,that how to set the frequency option needs to be carefully wrestle.The choice of frequeny should be extensive enough,so that it can cover a majority of interviewers' possible condition.The choice of frequency also should be enough sensitive,so that it can incarnate the difference of collected result.In this questionnaire,a question about frequency investigation is designed like this:
Whether you make mistakes because of chronic operation action.For example,you chronically use the shortcut key of other table software that lead to the confusion of the table?
A.Often have
B.Sometimes have
C.Not have
We see that,only the option "Not have" in three frequency option is very correct.It represents zero occuring rate.From "Not have" to "Sometimes have",to "Often have",these three options make interviewers feel the trend of frequency increasing.However,the changing process of this increasing by degrees is very undefined.In question,it doesn't provide interviewers a division point of "sometimes have" and "often have".So interviewers are forced to guess what frequency is "often" on earth.The result of this guess is disunion of using standard,the sensitivity of the option is very low.Every interviewer imagine a standard according to his own subjective estimation,and then answers this question according to this standard.So,for the same frequency,some interviwers choose "sometimes have",but some choose "often have",it's irresponsible to base on the conclusion of such data,even it's possible to make readers of investigation report make a wrong decsion because of the misdirecting of data.
6.Put an end to involuntary inducement
In user research,a side of user communication must make themselves keep a nonaligned positon all long,and don't let their own action and language affect the estimation of users.That is to say,should avoid producing inducement to users.The questionnaire is so.Though in the questionnaire investigation,there is no face to fact with interviewers,the word inducement also has the possibility to exsist,it also possibally has powerful effect on the choice of interviewers.Let us look at the last question of this questionnaire.
Can you solve the operation problem of software by telephone or email for your friends?For example,a fresh computer learners expect to know how to make some lines in the table sightless,and he calls you for help.
A.Can settle it breezily.
B.Need to first ask him whether he can find a certain command in some place,and then you can teach him
C.Can't speak clearly at all.
In this question,there exsists a impelling inducement to interviewers:ie,the example raised in the question.Although this example also give interviewers an inducement about the complexity of "solve the operation problem of software".When facing a comparatively uneasy question,a example will become an important reference factor to understand and consider questions,even it's a exclusive factor.Interviewers will naturally choose this example to pick out the option.Whether you are doing so?And we know,there is a great discrepancy between the complexity of problem in practical condition described by this question and the complexity embodied by the example.This excessively simple example will possibly make interviewers have wrong easy-deflection estimation of the true complexity of the question,so that lead to differece between the choice of qeustion option and when making decision on the condition of correctly know complexity.In this way,the investigation result also lose its original purpose.Inducement is mistake which are easily to make when uncareful.In most condition,the appearence of inducement is involuntary,butt because it's involuntary that it need us to try our best to avoid it.Without discovering inducement in time will make us absence of waking up to the warp that exsists in data and result.The deduction is made because of absence of this,the decision may be all wrong.In user research,we need at every time to check our own action and language,keep guard on inducement,and by mutual notice between accompaniers to avoid the appearence of the inducement.
Although this questionnaire has five questions,from every question design we find their different existent problem.It seems that designing questionnaire is simple.Actually not so,it can't be accomplished in an action.In order to gain good investigation effect,you should work hard on the questionnaire design.Although not like wrestle word by word,it should be tasted carefully.Otherwise,it's very possible to draw water with a sieve,spending time,money,manpower to complish data collection.However,as a result,because of the limitation of questionnaire design that lead to the practical data can't represent the data that the design expects.To be worse,if these collected wrong data are not woken up to be useless,and still is regarded as credible data that infect the decision of product development,marketing and so on.This sequent may be more and more grave.
Mr Lin is the memeber of UMBC alternation designing center.His teacher is Doc Andrew Sears,taking on the research work of available area.He has taken part in many item which is staked by USA State Natural Science Fund and large-sized corporation,has come out with a few learning thesis.He is the member of ACM and SIGCHI.He has worked in gainable research group of IBM.J.Waston.
The website UnderstandUsability.com is a communication platform which is pened for country craft brother that aiming at driving China availible