The Impact of Surface Co-location and Eye-tracking on Mixed Reality Typing

Abstract: Accuracy and speed are pivotal when typing. We hypothesized that the lack of tactile feedback on midair mixed reality keyboards may adversely impact performance. Our first experiment investigated providing tactile feedback to users typing in mixed reality by co-locating the virtual keyboard on a table or a wall. The keyboard was deterministic (without auto-correct), allowed mixed case typing with symbols, and relied only on the hand-tracking provided by a commodity headset's egocentric cameras. Users preferred and had the highest entry rate of 12 words-per-minute using a midair keyboard. Error rates were similar in all conditions. Based on user feedback, our second experiment explored ten-finger typing. We used a novel eye-tracking technique to mitigate accidental key presses. This technique was preferred for ten-finger typing and halved the number of times backspace was pressed. However, participants were faster using only their index fingers without eye-tracking at 11 words-per-minute.