user testing
Wednesday, January 30th, 2008I did user testing to three candidates with Samsung NV15 digital camera which is not very popular in compact digital camera market nowadays. What’s unique about this product is ‘multiple control buttons’ named SMART BUTTON on the back, in both vertically and horizontally. Overall online users’ verdict was either bad for too many confusing button’s or good for usefulness and even quickness of those buttons. Therefore, I decided to concentrate more on how users will grasp the sense of using buttons to manipulate several functions from easy level to rather difficult level.
Given tasks are taking picture of coin in both AUTO and P(Programmed) modes and save/protect the image.
[MD]=Mode Dial [A]=Auto [P]=Programmed [V]=Vertical Smart Button [H]=Horizontal SB [F]=Focus [FL]=Flash [S]=Size [Q]=Quarlity [PM]=PlayMode Button [PT]=Protect [WB]=White Balance
- go to ‘auto’ mode [MD][A]
- change the resolution/quality of photo (i.e. 1m to 5m/ normal to superfine)[H]3(3rd button from left)[V]4(4th from top)/[H]7(7th is ’shift’ to see hidden layer)[H]3[Q][V]5
- disable ‘flash’ mode [H]2[FL][V]5
- go to ‘auto macro’(flower icon) mode [H]1[F][V]4
- take a picture of a coin Press Shutter
- go to ‘view’ and ‘protect’ the image [PM][V]3[H]5[V]3
- go to ‘programmed’ mode [MD][P]
- change ‘WB- white balance’ from cloudy to whatever is appropriate under the circumstance. [V]2[WB]and select by sliding left - right
- change ‘ISO’ from 80 to auto [V]3[ISO][H]6
- diable ‘flash’ [H]2[FL][V]5
- auto macro(from AF) [H]1[F][V]4
- FIND a ‘timer’ mode which is hidden in another layer of function and set it to 2sec timer [H]7[H]4[V]3
- take a picture of a coin by using of timer
A. Predict:
- Problem of the appliance.–> In one word, this camera is ‘too much’ at every aspect. The camera has total of fourteen buttons on the back to execute any function. They are touch-sensitive but too ’sensitive’ when touched, causing inaccuracy. Users will find this very annoying when they attempt to navigate tons of features and functions.
- The typical ways that this task can be achieved, using a notation schema.–> Users should figure out how to execute any function clicking ‘cross point’. Once a user grasps the sense of touching the buttons in ‘proper’ way, they will also get the ‘next’ sense of ignoring inaccuracy of them. For example, the touch button tends to show very next function when pushed, however you should be courageous to press the exact function you desire to get no matter what.
- Demonstration Video
B. Observe:
- Users were not told about any information, that made most of them embarassed when they realized many buttons on the back. First user rushed to finish the tasks the most, but he managed to accomplish it in almost five minutes which is two minutes longer than my guess for average. Second user was Japanese who took the longest time to get it done. As soon as he noticed that this camera’s operation is not easy at all, he started feeling more nervous even about available language which wasn’t his first language.(Samsung NV15 doesn’t support Japanese.) It took him for more than ten minutes to finish. Interestingly, he got stuck in ’slide show’ mode when trying to save an image and ‘protect’ it while others didn’t go there at all. He spent five minutes in MANUAL MODE testing many buttons without realizing he’s in wrong menu. Third female user was winner. Her record for first mission was three minutes and forty three seconds.
- Most interesting thing was two of users kept touching LCD panel at the beginning. They seemed ‘touch-sensitive buttons’ also means that operation should be possible through ‘touch screen LCD’, as well.
- Second user attempts to use MODE DIAL even though the task had nothing to do with it. Now I think it’s reasonable to think both dial and buttons as related combination.
- User 1 - Task 1 video
- User 2 - Task 1 video
- User 3 - Task 1 video
C. Analysis
- Despite many other great features of this camera, SMART BUTTON turns out to be ‘not-smart’ by three users almost painful trials to accomplish two different type of shooting. It requires quite a bit of ‘learning curve’ to learn the following mechanisms: 1. selecting - by touching the button 2. confirming - by pressing 3.to move left /right by sliding finger 4. ’some’ sub menus are hidden in layer and it will be displayed when extension menu button is pressed.
- In detail, I found following inconsistancy in terms of unified interface. First, in [PM] play mode, it only shows icons, not the name of the function when the button is half-pressed in touch-sensitive way. This could be against the users’ expectation now showing the function name so that users should take a guess only by looking at the icon itself. In given task to protect the image, the icon appears to be a ‘key’, when pressed, then it shows next two steps. Requiring more than two steps to protect an image could be ‘too much’, as well.
D. Improvement
- Putting each function’s short name on buttons (i.e: F - Focus, Q- Quarlity)
- Indicating both Horizontal and Vertical rows by putting [V], [H]
- Differentiating ‘top’ and ‘bottom(hidden layer)’ row’s function
- On LCD screen, eliminating ‘current’ view which causes more confusion when users are navigating
Main idea to improve current UI of the camera is simply give more ‘kind’ things such as printed letter on top of each button. I could immediately notice that Samsung pushed more on design aspect, that’s why this model has very minimal printing on the back, which might give sophisticated ‘look’ of the camera, not user-friendly ‘function’.
First available user was third user of previous testing.
–>see video
It took far much less time to complete the improved-task than previous one, she felt more comfortable using ‘numbered’ and ‘printed-letter on top’ type of new buttons. One problem occurred was that she skipped pressing ‘view’ button to protect the image, as well as going into sub-menu by pressing ’shift’ button (this shows as escalator-looking icon, not the letter ’shift’). However, she immediately found ‘protect’ button because I altered it from ‘key-looking’ icon. It is interesting to find that few users could tell those picture icon to execute any function. Some got confused and lost confidence using them.
Second video shown is from first user of previous testing. He was able to finish the task in almost same amount of time that second user’s, but showed identical pattern in skipping ‘view’ button.















