This is one of the modules that require a lot of works (research, designing, redesigning) throughout the semester. We learned the different methods of user research and testing. The most interesting thing that I've learned is to design for individual. We learn to make user persona and do four pleasure analysis for that user. However, sometimes I still doubt the idea of designing for each individual. I believe that each individual is unique in taste - likes and dislikes - but how can we design a product for each of the billions of people in the world? Although we discuss in class that 'Apple' designs the user experience products that make us feel like each product is designed especially for 'you'. But it is not really the case for me. I have never been a fan of any of 'Apple' product and for me, their products are just massively-produced over-prized china made non-durable products. Of course, in this case, it shows why it is important to create the persona for anti-user and make sure that our product is not designed for these users.
We learn all the different techniques of finding out more about our users because we need to put in consideration of user's experience of the product in every stage of design process. A lot of time, I learn to put myself into the shoes of users, discarding my own designer's ego that is telling me to put more functions to make things 'cool'. However, I would argue that there is also 'luck' and 'time' factors in the design as well. A designer can make a perfect product/service that has taken consideration of user experience in every process but in the end, the design still failed. Maybe the failure is partially caused by the misinterpretation of 'need' and 'want' from the target audience that we do research on.
All in all, designing a user experience is an art itself. There is no specific guidelines or processes that will guarantee a successful experience. However, I believe a good balance between user research findings and designer's professional expertise will lead to a good experience design.
Thursday, April 24, 2008
Reflections on User Research Smoke & Mirrors article
The five-part article on "User Research Smoke & Mirror" debates on the usefulness of scientific research to lead the design process.
In the first part of the article the author argues that the scientific approach to design process is idealistic and can hurt the practice of design. Since individuals are subjective, putting quantitative results to the subjective studies and misinterpreting these results will mislead the design changes. A good expert with deep design experience and good instincts could have rather avoided the design errors by him/herself.
I agree with the author to an extent. On one hand, I do agree that it is not 100% possible to put assign a number to user's emotion. All scientific research use preset questions that do not allow flexibility in user's feedback. If you interview 10 people, you will get 10 different answers with variations in intensity. However, scientific research constrains what the respondents can choose within a specified scale. On the other hand, I do think that scientific research, while not a good tool as a foundation of design process, could still be helpful to give general trend how a user feel about the design. Since expert designers are also individual with subjective thoughts, sometimes they could be so engrossed in their designs that they think they are perfect. The importance lies in where the designer has to draw a line analytically when to follow the research findings or when not to.
The second part of the article mainly talks about the usefulness of eye tracker to help with website designs. I do agree with the author that eye tracker only gives information of 'where' the user is looking at but not 'what' and 'why'. While pure scientific measure such as this eye tracker might not be of much help to the design process, a combination of the scientific measurements and qualitative measurement such as "think aloud" technique will be a powerful tool to have insights into user's reactions to the design.
The third part of the article reveals the industrial truth surrouding the research. The author explains why we need to conduct research methodologies for websites even though the flaws of a website is so distinct towards an expert. I would agree that it is helpful to use research data as a tool to justify the non-user stakeholders or big bosses of the design proposal because they would prefer to see a concrete figure rather than listening to the designer's explanation. However, I also feel that it is a total waste of resources just to cook up a figure for the justification.
The fourth part talks about how the professional researchers tend to make up the environments for the users to be in. Therefore, their date become more artificial than actual. However, their services are still good not from the research methodologies but from the expertise and experience that those professionals themselves. If this is the case, then why do we need the research after all? In this part, I think the examples that the author showed are too extreme to one end. I believe there are good research that are done in the controlled environments as well.
The last part does state an need for non-scientific research to guide the design process. Again this is very subjective. Just as I mention above, the designers have to be careful of whether the feedback should be incorporated into the design process or not, from their careful analysis of the feedback and situations themselves.
My overall reflection on the article is that some parts are well-argued, such as "design as politics", explaining why there is a need to conduct the quantitative research. However, a thing to be careful about is that not every single feedback from the qualitative research should be incorporated into the design process.
In the first part of the article the author argues that the scientific approach to design process is idealistic and can hurt the practice of design. Since individuals are subjective, putting quantitative results to the subjective studies and misinterpreting these results will mislead the design changes. A good expert with deep design experience and good instincts could have rather avoided the design errors by him/herself.
I agree with the author to an extent. On one hand, I do agree that it is not 100% possible to put assign a number to user's emotion. All scientific research use preset questions that do not allow flexibility in user's feedback. If you interview 10 people, you will get 10 different answers with variations in intensity. However, scientific research constrains what the respondents can choose within a specified scale. On the other hand, I do think that scientific research, while not a good tool as a foundation of design process, could still be helpful to give general trend how a user feel about the design. Since expert designers are also individual with subjective thoughts, sometimes they could be so engrossed in their designs that they think they are perfect. The importance lies in where the designer has to draw a line analytically when to follow the research findings or when not to.
The second part of the article mainly talks about the usefulness of eye tracker to help with website designs. I do agree with the author that eye tracker only gives information of 'where' the user is looking at but not 'what' and 'why'. While pure scientific measure such as this eye tracker might not be of much help to the design process, a combination of the scientific measurements and qualitative measurement such as "think aloud" technique will be a powerful tool to have insights into user's reactions to the design.
The third part of the article reveals the industrial truth surrouding the research. The author explains why we need to conduct research methodologies for websites even though the flaws of a website is so distinct towards an expert. I would agree that it is helpful to use research data as a tool to justify the non-user stakeholders or big bosses of the design proposal because they would prefer to see a concrete figure rather than listening to the designer's explanation. However, I also feel that it is a total waste of resources just to cook up a figure for the justification.
The fourth part talks about how the professional researchers tend to make up the environments for the users to be in. Therefore, their date become more artificial than actual. However, their services are still good not from the research methodologies but from the expertise and experience that those professionals themselves. If this is the case, then why do we need the research after all? In this part, I think the examples that the author showed are too extreme to one end. I believe there are good research that are done in the controlled environments as well.
The last part does state an need for non-scientific research to guide the design process. Again this is very subjective. Just as I mention above, the designers have to be careful of whether the feedback should be incorporated into the design process or not, from their careful analysis of the feedback and situations themselves.
My overall reflection on the article is that some parts are well-argued, such as "design as politics", explaining why there is a need to conduct the quantitative research. However, a thing to be careful about is that not every single feedback from the qualitative research should be incorporated into the design process.
Final Project
Brigadier
Who are we?
We are a subsidiary of DELL, who makes use of the existing DELL business model and tape it onto the production of hand phone, targeting at military personnel.
Our team comprises 4 members from different backgrounds, namely Yi Arn, Wei Wei, Weiye and myself, Nilar.
What we have done?
During the initial stage of the project, we had done user research and defined our target audience and come up with user profile and user personas. From there, we move on to do four pleasure analysis of our product on the personas - advanced user, novice user and anti-user.
After we have done much research on our target audience, we moved on to card-sorting and information organization of our product, which is a website offering the service of customizing your own hand phone.
After the card-sorting, we come up with the different fidelity prototypes and the endless process of designing, gathering feedback, redesigning, gathering feedback follows.
I will not talk much about the detailed tests and research that we have conducted for the design process, as they can be found in full details in our report.
My reflection
Looking back at the report and what we have done for the project, I realize that it has been a really long journey of designing and redesigning. As a novice designer, we all tend to be out of track during our design process. There have been a lot of time when we just want to discard our idea and come up with a totally new idea to work on because we feel that we can't figure out how to continue our current one. However, throughout the process, we also tried to stick back to our original objective for the project and moves on with our initial idea.
In the class, I have seen some groups that really discard their original idea and come up with a new product. I believe that the journey to become a good user experience designer is to be able to make correct judgements on the users' feedback. I believe that if we ask the users what will increase their positive experience towards a product, most of them will reply 'the more the better'. I have seen the group working on breakfast delivery service totally change their target audience and services after the user research. I have also seen some group starting to create a totally new version of prototype each time after they've done user testing with it.
A lot of time, I have also wanted to give a little bit more to the users. I think it's the designer's ego and pride that we need to learn to discard during the design process. After the user testing from the low fidelity prototype, we are to make a decision whether to discard the 'slider' and 'calm shell' design phones because they connote 'weak' design phone for military. However, I had argued that since we already have the design for them, why don't we just leave the offer there for the non-military audience. This reminds me of the spicy chili crap example given in class. In the end, our team decided to offer products specifically for our target audience only. This is what the 80-20 rule is about.
Another lesson that I learnt from the project is how much changes we should do to our prototype after each user testing. If the design process is an endless cycle and the product should improve a bit after each cycle of designing and redesigning, how do those groups that come up with a totally new prototype, with all different looks and features, etc continue this spiral process of design? I learn to be always aware that each small change can lead to a big difference, thus before doing any changes, we have to justify them professionally.
Who are we?
We are a subsidiary of DELL, who makes use of the existing DELL business model and tape it onto the production of hand phone, targeting at military personnel.
Our team comprises 4 members from different backgrounds, namely Yi Arn, Wei Wei, Weiye and myself, Nilar.
What we have done?
During the initial stage of the project, we had done user research and defined our target audience and come up with user profile and user personas. From there, we move on to do four pleasure analysis of our product on the personas - advanced user, novice user and anti-user.
After we have done much research on our target audience, we moved on to card-sorting and information organization of our product, which is a website offering the service of customizing your own hand phone.
After the card-sorting, we come up with the different fidelity prototypes and the endless process of designing, gathering feedback, redesigning, gathering feedback follows.
I will not talk much about the detailed tests and research that we have conducted for the design process, as they can be found in full details in our report.
My reflection
Looking back at the report and what we have done for the project, I realize that it has been a really long journey of designing and redesigning. As a novice designer, we all tend to be out of track during our design process. There have been a lot of time when we just want to discard our idea and come up with a totally new idea to work on because we feel that we can't figure out how to continue our current one. However, throughout the process, we also tried to stick back to our original objective for the project and moves on with our initial idea.
In the class, I have seen some groups that really discard their original idea and come up with a new product. I believe that the journey to become a good user experience designer is to be able to make correct judgements on the users' feedback. I believe that if we ask the users what will increase their positive experience towards a product, most of them will reply 'the more the better'. I have seen the group working on breakfast delivery service totally change their target audience and services after the user research. I have also seen some group starting to create a totally new version of prototype each time after they've done user testing with it.
A lot of time, I have also wanted to give a little bit more to the users. I think it's the designer's ego and pride that we need to learn to discard during the design process. After the user testing from the low fidelity prototype, we are to make a decision whether to discard the 'slider' and 'calm shell' design phones because they connote 'weak' design phone for military. However, I had argued that since we already have the design for them, why don't we just leave the offer there for the non-military audience. This reminds me of the spicy chili crap example given in class. In the end, our team decided to offer products specifically for our target audience only. This is what the 80-20 rule is about.
Another lesson that I learnt from the project is how much changes we should do to our prototype after each user testing. If the design process is an endless cycle and the product should improve a bit after each cycle of designing and redesigning, how do those groups that come up with a totally new prototype, with all different looks and features, etc continue this spiral process of design? I learn to be always aware that each small change can lead to a big difference, thus before doing any changes, we have to justify them professionally.
Assignment 3 - Emotion Design Probe
For the assignment 3, we are tasked to improve the learning experience of students in Lecture Theatres. In order to gain sufficient insights into the user experience of students in lecture theatres, our group members decided to use 3 different approaches - Cultural Probe, Laddering and Ethnographic study.
Cultural Probe
To perform cultural probe, one of our members, Weiye, took a convenient sampling of 3 NUS friends and asked them to give a probe on a notebook. The duration of the experiment is 1 week.
Our findings are summarized into 2 main categories: non-lecture related and lecture related experience.
The non-lecture related experiences indirectly affect the overall experience of learning in lecture theatres. They include 1) tiredness due to the long journey to and from campus, 2) the lack of seats on public transports, 3) insufficient sleep in the previous night, and 4) the lack of time to have breakfast in the morning. As a result, students are not able to concentrate on the lectures, hence affecting their learning experiences. Another non-lecture related experience is the day of the week when the lecture is held. We gained feedback from our participants that the 'Monday blues', 'mid-week crisis' and 'TGIF-syndrome' also affect the experience of learning. In additions, students are unmotivated to attend the class right after public holidays. On the other hand, peers/friends also play a role in influencing you to skip lectures.
The lecture related experiences are broken down into 5 sub-categories.
Laddering
In this approach, we interviewed a third year Arts student a series of 'why' question to sieve out the roots of the problem.
Big size -> Students sitting scattered -> Talk very loud -> Distractions -> Can't make full use of tuition
From the interview, we realized that the root factor contributing to the overall user experience of learning in lecture theatres is related to the amount of tuition paid (cost) and the perceived amount of learning (benefits).
Ethnography Study
For this approach, one of our group members, Wei Wei, crashed a lecture to observe and take notes of the whole process of learning in the theatres.
The behaviors observed are as follow:
1) Students brought own notebook, surfing the net or chatting on instant messenger during lecture.
2) Most students sit at the back seats, talking amongst themselves. The front rows are empty.
3) Some students eat and drink in the lecture.
4) Some students are sleeping during lecture.
5) Sometimes, the lecture notes and writing materials fell or were almost falling off the writing tablets.
6) The lecturer's voice was drowned in a hissing noise produced by the loudspeakers.
7) The projector screen was out of focus, with the lecture theatre's lighting casting a glare on it.
8) The flipping and jostling lecture notes and paper were observed.
9) Lecturers give too short a pause for students to copy notes, as evident from some turning their head to their fellow course mates to copy what they missed out, or too long a pause, as evident when students start talking amongst themselves.
10) Questions asked by students from front rows cannot be heard from the back.
Findings
From the three approach, we've concluded 3 common problems of learning in lecture theatres.
1) The lecture theatre's design - the theatres are generally too big for most classes, resulting in empty/half-filled lectures. This leads to students sitting at the last few rows and starting to talk amongst themselves.
2) Lecturer factor - the lecturing style, accents of lecturers affect the user experience of learning. The lack of 'crowd control' by the lecturers may negatively impact the learning experience of students who genuinely want to attend the lectures.
3) Student factor - Many students are affected by their fellow students' improper and insensitive behaviors/attitude during/towards lectures.
Solutions
Based on the findings, we proposed that
1) the lecture timing should not coincide with peak hours (office hours) as this may indirectly affect the traveling to and from campus for lectures. Back-to-back lecture timing should also be avoided so that students can have time to have breakfast or lunch.
2) On the student part, they should be nurtured to have good time management skill and positive attitude towards learning.
3) Students need to be aware of and considerate to other students who are attending the lectures, by not talking, eating, or doing anything disruptive to the lecture.
4) Big-sized lectures should be replaced with seminar style teaching with maximum capacity of 30 students per class. The smaller class will allow lecturers to perform better crowd control.
5) Lecturers should also improve their lecturing style by making the lecture more interactive. Accented English is acceptable as long as it is comprehensible.
My reflection on the assignment
User experience varies greatly among individuals. I think it is not possible to ask users what can create positive experience for them because they will just ask for the best thing, which is idealistic.
Now looking back at our proposal, I realized that most of the points that we proposed are idealistic. If they could be achieved, the problems would not exist in the first place. Therefore, I have to re-analyze whatever we have done for this assignment. If subjective interviews/collections of feedback such as laddering and culture probe are not very helpful feedback because the users will tend to ask for the perfect solution, which could hardly be achieved, would the 'objective' observation method, such as ethnography study, helps us in improving the user experience while we would only know 'what' is happening but not 'why' the students are behaving this particular way?
My answer is that each of them, standing alone, might not yield very helpful information towards improving the user experience. But a combination of them might be very useful. I think what went missing in our 3 approaches is the link among the three. We did the 3 approaches independently. And it could be that the sample size is too small. Therefore, I find that the findings from each approach does not concretely support one another.
Another thing that we have to be careful about is that we need to draw a clear line (as a professional) between what is improvable and what is idealistic.
Cultural Probe
To perform cultural probe, one of our members, Weiye, took a convenient sampling of 3 NUS friends and asked them to give a probe on a notebook. The duration of the experiment is 1 week.
Our findings are summarized into 2 main categories: non-lecture related and lecture related experience.
The non-lecture related experiences indirectly affect the overall experience of learning in lecture theatres. They include 1) tiredness due to the long journey to and from campus, 2) the lack of seats on public transports, 3) insufficient sleep in the previous night, and 4) the lack of time to have breakfast in the morning. As a result, students are not able to concentrate on the lectures, hence affecting their learning experiences. Another non-lecture related experience is the day of the week when the lecture is held. We gained feedback from our participants that the 'Monday blues', 'mid-week crisis' and 'TGIF-syndrome' also affect the experience of learning. In additions, students are unmotivated to attend the class right after public holidays. On the other hand, peers/friends also play a role in influencing you to skip lectures.
The lecture related experiences are broken down into 5 sub-categories.
- The lecturer - a good-looking lecturer may attract more students to attend the lecture regularly. Like-wise, the reverse is true.
- The lecture content - affects the overall user experience of learning in lecture theatres.
- Lecturing style - Participants generally feel a decline in learning experience if the lecturing style is monotonous, lecturers reading off the slides, and lecturers having incomprehensible accents.
- Lecture timing - Participants do not like to attend lectures that are too early or too late.
- Peer influence - reportedly reduces other students talking during lectures, skipping lectures or sleeping during lectures.
Laddering
In this approach, we interviewed a third year Arts student a series of 'why' question to sieve out the roots of the problem.
Big size -> Students sitting scattered -> Talk very loud -> Distractions -> Can't make full use of tuition
From the interview, we realized that the root factor contributing to the overall user experience of learning in lecture theatres is related to the amount of tuition paid (cost) and the perceived amount of learning (benefits).
Ethnography Study
For this approach, one of our group members, Wei Wei, crashed a lecture to observe and take notes of the whole process of learning in the theatres.
The behaviors observed are as follow:
1) Students brought own notebook, surfing the net or chatting on instant messenger during lecture.
2) Most students sit at the back seats, talking amongst themselves. The front rows are empty.
3) Some students eat and drink in the lecture.
4) Some students are sleeping during lecture.
5) Sometimes, the lecture notes and writing materials fell or were almost falling off the writing tablets.
6) The lecturer's voice was drowned in a hissing noise produced by the loudspeakers.
7) The projector screen was out of focus, with the lecture theatre's lighting casting a glare on it.
8) The flipping and jostling lecture notes and paper were observed.
9) Lecturers give too short a pause for students to copy notes, as evident from some turning their head to their fellow course mates to copy what they missed out, or too long a pause, as evident when students start talking amongst themselves.
10) Questions asked by students from front rows cannot be heard from the back.
Findings
From the three approach, we've concluded 3 common problems of learning in lecture theatres.
1) The lecture theatre's design - the theatres are generally too big for most classes, resulting in empty/half-filled lectures. This leads to students sitting at the last few rows and starting to talk amongst themselves.
2) Lecturer factor - the lecturing style, accents of lecturers affect the user experience of learning. The lack of 'crowd control' by the lecturers may negatively impact the learning experience of students who genuinely want to attend the lectures.
3) Student factor - Many students are affected by their fellow students' improper and insensitive behaviors/attitude during/towards lectures.
Solutions
Based on the findings, we proposed that
1) the lecture timing should not coincide with peak hours (office hours) as this may indirectly affect the traveling to and from campus for lectures. Back-to-back lecture timing should also be avoided so that students can have time to have breakfast or lunch.
2) On the student part, they should be nurtured to have good time management skill and positive attitude towards learning.
3) Students need to be aware of and considerate to other students who are attending the lectures, by not talking, eating, or doing anything disruptive to the lecture.
4) Big-sized lectures should be replaced with seminar style teaching with maximum capacity of 30 students per class. The smaller class will allow lecturers to perform better crowd control.
5) Lecturers should also improve their lecturing style by making the lecture more interactive. Accented English is acceptable as long as it is comprehensible.
My reflection on the assignment
User experience varies greatly among individuals. I think it is not possible to ask users what can create positive experience for them because they will just ask for the best thing, which is idealistic.
Now looking back at our proposal, I realized that most of the points that we proposed are idealistic. If they could be achieved, the problems would not exist in the first place. Therefore, I have to re-analyze whatever we have done for this assignment. If subjective interviews/collections of feedback such as laddering and culture probe are not very helpful feedback because the users will tend to ask for the perfect solution, which could hardly be achieved, would the 'objective' observation method, such as ethnography study, helps us in improving the user experience while we would only know 'what' is happening but not 'why' the students are behaving this particular way?
My answer is that each of them, standing alone, might not yield very helpful information towards improving the user experience. But a combination of them might be very useful. I think what went missing in our 3 approaches is the link among the three. We did the 3 approaches independently. And it could be that the sample size is too small. Therefore, I find that the findings from each approach does not concretely support one another.
Another thing that we have to be careful about is that we need to draw a clear line (as a professional) between what is improvable and what is idealistic.
Subscribe to:
Posts (Atom)