Comparison of menu techniques
...culty: Medium – High Real World Analogous: Novice user. New System; Users learns with need. Reason: This task was chosen to test the user’s ability to find elements when the user does not have pre-information on node locations. Above three conditions were applied to this task to exhaust the physical space of the menu items in the widget. T3. Select a menu item from a deepest level where the first item is given where items on particular axis. (e.g. Find “Apple1” ; Start at “Food”; Direction: On-Axis). C7 Meaningful; Depth 4; Breadth 8; On-axis menu path (N, S, E, W) C8 Meaningful; Depth 4; Breadth 8; Off-axis menu path (NE, NW, SE, ...) C9 Meaningful; Depth 4; Breadth 8; On-axis & off-axis mixed menu path Level of difficulty: Medium Real World Analogous: Experienced user. User is on way to becoming expert. User has some knowledge of system. Reason: This task was chosen to test the user’s ability to find elements when the user only knows the first element and has information about the directions/axis(and hence user has to filter out menu items not on given axis/direction). The conditions were chosen to test if the direction of menu(axis) makes a significant difference in error rate and completion time. The same condition will also help us test if user can filter out extra information and speed up the task. Design: (See Pici: Experiment) A within-participants design was used. Subjects performed multiple trials on three different tasks for each technique. Each block was a series of 18 trials; each session was a series of three blocks. Conditions were presented in the same order in each block on purpose. This is because previous research has already shown that there is transfer effect between conditions and we wanted to test that rate of it for different widgets. Trials between blocks were randomized. The trials were chosen to exhaust all directions of menu widgets. The order of widgets was switched across the users to achieve basic counterbalancing. For each trial, a task instruction was given to the user and users response was shows as user performed the action. We allowed user to continue on mistakes to reduce confusion and to learn what user does after making a mistake. System noted down the failed attempt. Incorrect answers made the response label color red, whereas correct response made the label color green. Dependent variables were task completion time & error rate. Task completion time was calculated from user’s first click to show the menu until click on a leaf node item. We ignore the reaction time as we assume it to be same across all widgets for each user. Independent variables are menu depth, menu breadth & direction (on-axis, off-axis). Each participant performed the entire experiment at one sitting, including breaks, in approximately 1 hour 15 minutes. Participants were explained the purpose before the experiment and were told that no assistance will be provided during the experiment. Before starting the actual study, a pilot study was performed with 1 repetition of each condition; 3 blocks of all conditions. In summary, the design was setup in following way: 2 Participants x 3 Techniques x 3 Tasks x 3 Blocks x 3 Conditions x 6 Trials = 972 selections in total Section 3: Hypotheses Based on initial analysis, we found that it is very hard to apply Fitt’s law or Accot’s law to given menu techniques. This is because these laws rely heavily on target width, path width & distance of the target item. In our case however, the techniques do not require the user to hit/cross the desired item to go to next level of details, instead, it simply required mark in the direction. Based on various other factors, we derive the following Hypothesis: H1: Sunburst marking technique will have shortest task completion time followed by Expanded and single stroke(for all tasks) • Recall instead of recognition. • Better affordance (sunburst & expanded) • Users have limited storage capacity of short-term memory (7 +/- 2). Sunburst technique fits best by showing as much detail and hence less to remember. H2: Single stroke marking technique will have smallest error rate for T1(Task 1) and highest for other tasks. • Cluttering of data. size and complexity can be intimidating, so for T1, when user knows the path, less chances of making mistake compared to other 2 techniques. • For T2 & T3, since user can only see one level of details, user is likely to make more mistakes compared to expanded and sunburst where user can see more than one level. H3: Sunburst will have higher long-term learning effect. We are not very interested in short-term learning effect. • Sunburst gives user preview of many levels of data and hence user are likely to take advantage of this, specially for tasks similar to T2 & T3. H4: Users are likely to make fewer mistakes when completing on-axis tasks, compared with off-axis tasks. For successful task completions, time for completion should be largely independent of widgets. This is because each will require user to do similar strokes. H5: As menu depth increases, the error rate and task completion time will increase significantly. Due to nature of sunburst, sunburst will be least affected. H6: Breadth of menu items will have a significant effect on task time completion for all items, but most noticeable for sunburst followed by Expandable. For all 3 tasks, I would be interested in finding out the learning rate across widgets and blocks. I would also be interested in error rate and task completion time across conditions and widgets. Section 4: Results and analysis (Due to space constrain, I have attached all my excel figures in the appendex) Accuracy/Errors: Task 1: The overall F statistic was insignificant. Unexpectedly, widgets did not make much difference in error rate. As expected (H1) Singe stroke had lowest errors followed by Expanded & Sunburst. Also, condition (menu breadth) did not have a significant effect as users knew the menu path. Task 2: As expected, condition(depth) had the most effect on error rate(t-group A,B). Error rate increased fastest for single stroke followed by Expanded & Sunburst. The last 2 had somewhat similar error rate (see Fig. Error1) which can be explained by the trade-off between cluttered data & more visibility. Un expectedly, Expanded & Single had similar error rates as conditions increased (see Fig Error1). Task 3: The overall F statistic was (F =3.16, p =<.0001) as expected in H4. Condition, as expected again, had a huge effect on error rate (F =28.30, p = 0.0001). As predicted, error was lesser when user knew the axis(C7,C8) then when users did not know the axis(C9). T-grouping for on-axis condition was A and that for the other two was B. Task Time Note: (1) Reported avg. time is only for tasks that were done correctly, (2) H1, H2,.. etc refers to hypothesis. Task 1: The overall F statistic was significant (F = 13.99, p = 0.0001), indicating that the model as a whole accounts for a significant portion of the variation in Yield and that you may proceed to tests of effects. ANOVA analysis showed that widgets have a significant effect (F = 11.84, p = 0.0001) on task completion time. Strangely, at 95% confidence Single ...