本科生毕业设计 (论文)

智能手表的非接触式备择交互模式的设计

Designing Alternative Contact-free Control Modalities for Smart Watches

教学单位计算机科学与技术

姓名欧长坤

学号 201231102123

年级 2012

专业计算机科学与技术

导师陈雅茜

职称副教授

联合导师 Andreas Butz (University of Munich)

职称 Professor

2016 年 5 月 18 日

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Contents

摘要 1

Abstract 3

1 Introduction 5

1.1 HCI & Wearable Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.2 Subject Signicance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.3 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3.1 Gesture Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3.2 Extension Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Interaction Techniques on Apple Watch 9

2.1 Traditional Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.1 Tap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.2 Swipe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2 Special Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.1 Digital Crown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.2 Force Touch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.3 Haptic Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.3 Other Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.3.1 Side Button . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.3.2 Voice Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3 Alternative Design of Contact-free 13

3.1 Interaction Pattern Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.1.1 Tap & View Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.1.2 Swipe & Continuous Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.1.3 Force Touch Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.1.4 Haptic Feedback Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.2 Completeness of Alternative Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4 System Design 17

4.1 Framework Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.1.1 LeapMotion & LeapJS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.1.2 watchOS & WatchConnectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4.2.1 Communication Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4.2.2 Client Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4.2.3 Server Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.3 Communication Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.4 Demonstrate Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

5 Implementations 21

5.1 Development Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

5.1.1 Conguration of Local LeapMotion . . . . . . . . . . . . . . . . . . . . . . . . . . 21

5.1.2 watchOS Networking Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5.2 Server Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5.2.1 Tap and Force Touch Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

5.2.2 Two Fingers Swipe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

5.2.3 Grab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

5.3 Client Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

5.3.1 View Controller Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

5.3.2 Communication Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

6 User Study 27

6.1 User Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

6.2 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

6.3 Analysis and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

6.3.1 Statistics of Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

6.3.2 Social Acceptable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

6.3.3 Comfortable Rate of Gestures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

6.3.4 Gesture Intuition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

6.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

7 Future Works 33

7.1 Weakness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

7.1.1 Hardware Related . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

7.1.2 Interaction Related . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

7.2 Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

7.2.1 Recognition Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

7.2.2 IPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

7.3 Closing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Appendix A Licence 37

Appendix B Questionnaire - Part A 39

Appendix C Questionnaire - Part B 41

Bibliography 43

Acknowledgments 47

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

摘要

本文以现有智能手表产品的代表 Apple Watch 为例，对 Apple Watch 上的交互模式

的优缺点进行了全面的分析，并据此给出了一套非接触式的备择设计。其配合了 Haptic

Engine 对用户的直观震动反馈，完成了对基础点按、滑动、Digital Crown、Force Touch

等系列原生交互的非接触式手势交互设计，此设计将嵌套两步逻辑的十种原生交互简化

为了单步逻辑上的八种备择交互，且消除了接触式交互的依赖，解决了现有交互中对双手

依赖的缺陷，并同时说明了给出的备择设计的交互完备性。通过用户调研，对本套交互方

式的设计进行了评估，结果显示该方案的操作逻辑和舒适度良好。在最后，本文对设计的

硬件结构、交互方式和系统架构的现有缺陷进行了讨论，并从中得到的启示，给出了可行

的解决思路。

关键词智能手表；非接触式；手势交互；备择设计

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Abstract

In this thesis, we explore Apple Watch as the representative of smart watches, and

analysed the advanteges and disadvanteges of interaction pattern in Apple Watch with

watchOS 2. According to this, we introduce a contact-free alternative design for smart

watch interaction.

This design, combine with the Haptic feedback, convert the basic tap, swipe, Digital

Crown, Force Touch and etc. native watch interaction to a contact-free gesture inter-

action. It simpleed ten native interaction with two-step logic to only eight alternative

interaction with sigle-step logic, eliminated contact-required interaction method, solved

the bimanual interaction within smart watches, and explained the completeness of this

alternative design.

Through user study, we evaluate these alternative designs and the result shows the

interaction operate logic and gesture confortable all acceptable.

Finally, we discussed the weakness of current hardware structure, interaction design

and system architecture. Trough this, we proposed a few implications for the future im-

provements.

Keywords Smart Watches; Contact-free; Gesture Interaction; Alternative Design

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

1 Introduction

With 30-year rapid development of computer chip technology, wearable devices sprang up in the

2010s, mainly represented by smart watches and smart bands in civil electronics market. With regard

to smart bands, they are only collecting data of users when wearing them. In essence, there does not

exist direct interaction between them and users. As for smart watches, despite that they are installed

with a large number of operating systems with simplied and appropriate designs, there still exist

performance weaknesses, thus bringing great challenges and opportunities for the software design of

wearable devices [1].

1.1 HCI & Wearable Computing

Since the middle and late 1990s, the concept of wearable devices has gradually come into people’

s view.

Wearable devices range from simple input devices(such as smart bands) to complicated devices with

operating systems and communication functions. Wearable devices are designed to provide computing

devices more intimate than smart phones for human beings. In essence, wearable computing is based on

this kind of small and wearable computers. “Always-on”is a distinctive feature of wearable computing.

Wearable devices are worn on a certain part of users to perceive their environment and observe their

behavior.

Wearable computing aims at providing people with personal assistant services, such as conve-

nient access to the Internet, event notication and information collection. This ubiquitous computing

technology based on wearable computing integrates multi-modal interaction, context-interactive model

and augmented reality into one, so it is the ultimate task desired in the eld of human-computer

interaction [2, 3].

However, limited to current level of science and marketing, these researches are still on a primary

stage. Above all, when reecting on the reasons for adding a device on themselves, people would

nd there it makes little sense to wear them, so wearable devices could be easily substituted. Even so,

researches on human-computer interaction of wearable devices are still on the way and have gone ahead

of electronics. Although there is still a long way to go to realize these designs and that they might

never be put into used, these research methods and concepts arising from them are of signicance for

people to continue pondering [

4] and reecting and promote the progress of the industry step by step.

1.2 Subject Signicance

Nowadays, the development of smart phones has been in a downturn, but wearable devices fail

to gain great momentum as desired. Regardless of their acceptability, usability and pricing, smart

watches do not have obvious signicance of existence, for their usages are much related to the contexts

they are in.

Therefore, the following issues must be considering carefully before design alternative interaction:

1. How to insure interaction correspond human intuition and comfort level?

2. How to erase sense of a separate existence(Social Acceptability)?

3. How to balance aordance and system complexity?

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

This thesis explore Apple Watch as the representative of smart watches, considering watchOS

features, designed a alternative interaction modality which can be contact-free for watch interaction.

Within this interaction pattern, user target can be more signicant, smart watch application scenarios

gains its extend and the complexity of interaction logic can be simplied so that the meaning of smart

watches will be indirectly increase.

1.3 Related Works

Smart watch interface is the most intuition part of the interaction [5], research on interfaces has

been more mature and in-depth [6, 7]. Apple Watch as a representative is the most simple, excellent

interfaces, and its Human Interface Guidelines [8] has become the standard guidelines of smart watches.

However, even in Apple Watch, user still need two hands to performing their actions. So, if we

have to redesign the whole interaction and release the bimanual interaction required on smart watches,

then we have to discuss gesture techniques due to we only have one hand to perform interactions at

this time.

1.3.1 Gesture Interaction

Gesture as a technique in human-computer interaction has been enduring, it can be cetegorise as

2D gestures and 3D gestures. For 2D gestures there are $1 [9] and $n [10] algorithms are maturing and

the main idea is resampling gesture point in same interval, then use 2-norm to caculate and compare

gesture series. For 3D gestures, the hardest part is to conrm where gesture starts and ends, however

most gesture recognize algorithm always avoid this problem by using recognize a specic gesture or

construct a state machine of gesture [11–15]. Fortunately, interaction on smart watches always after

user rise their hand, all information output through display ensures a fact that the amplitude of arm

could not too much or it will abstract users. Thus, interactions on smart watches could be gesture

manipulated but not identical.

L. Christian et al. [16] propused a complex gesture set of pinch for user to operate smart watches.

Although it’s purpose as same as ours, but their pinch gesture required redesign whole interaction logic

on smart watches, basiclly, the circumstances is unrealistic.

For research on contact-free interaction, [17] use a RGB camera attached on Google glass and

applied vision method to recognize gesture, but this method is only appropriate for classes user not

everyone. Kerber et al. [18] even expect to use diverse ortation of arm.

In these research, the interaction gains its augmentability, nonetheless they have ever prepense

these techniques applicable existence form.

1.3.2 Extension Interaction

Some research inspire us gives various idea, they expand the interaction outthrough watch itself,

they put interaction around the watch surface [19,20], band [21], even skin [22]. [23] puts sensors around

the watch and make it recognize gesture over the watch surface.

These interaction extension break through the limits of screen size though, but its still a bimanual

interaction, still tiredness and inconvenience for users.

This extension of interaction isn’t limited on smart watch, [24] augmented the smart phone inter-

action by smart watch, and [25] give us an implication for use watch position in space, essentially, it a

kind of Virtual Reality development. By this, we gains the multiplicity of watch state and even smart

phone and watch works simultaneously.

In summary, although the interactive mode research coverage broad in smart watches, but these

existing research on smart watches re generally avoided and did not consider the following questions:

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

• The theoretical interaction techniques could not compatible with existing smart watch interactive

mode which is not universal.

• These techniques are not broad enough, most of them didn’t make a full analyze on a series of

products and the design itself always give up on aordance.

• Most theoretical techniques are suitable for bimanual interaction, there is no obvious context for

increasing user stickiness on smart watches;

• Do not have much considering of the relationship between interaction mode and user interfaces, on

the one hand due to the lower social acceptability, and the other hand is because of the interactive

logic not too strong, learning costs of users too high.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

2 Interaction Techniques on Apple Watch

Most interactive mode in Apple Watch follows smart phone touch screen interactive mode [8]. To

design a contact-free alternative interactive mode on Apple Watch, we must analyze and clearly gure

out extant the advantages and weakness of interactive mode in Apple Watch.

2.1 Traditional Techniques

Apple Inc. believe the traditional multi-touch techniques on a small screen is severely aected user

experiences. Therefore they didn’t put multi-touch into Apple Watch. Besides, to operate object on

smart watch screen, user only have the basic tap gesture and swipe up, down, left and right for these

four directions, as shown on Figure 2.1

Figure 2.1: Traditional Techniques

2.1.1 Tap

Ordinary single tap on smart watches is as same as its on the smart phones. This tap gesture as

an human-computer interaction techniques has been accepted by the public for a long time, and this

natural technique gains its lowest cost among the most technologies.

However, the usability of this interaction will be reduced when the screen size reduced, in another

words, we are approaching the limits when we reduce the material size and move its position on our

body. According to FFitts’ Law [26]:

T = a + b log





2πe(σ

− σ

)

+ 1



(2.1)

where σ is the standard deviation of touch position distribution, σ

is the absolute accuracy of the

nger input, A is a distance as the start point to the target center.

Considering FFitts’ Law, A will be increase when screen size limited on the one hand, and σ will

not large than its on smart phone screen, on the other hand σ

will not change when we consider them

in same object.

Thus, T value will be signicantly larger, which means tap interaction gains poor usability on

smart watches.

2.1.2 Swipe

Continuous tap generate swipe gesture interaction. To process the weakness on smart watches,

Apple Watch as much as possible to make the interaction to be handle be swipe gestures.

Image Source：https://developer.apple.com/watch/human-interface-guidelines/

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Horizontal direction view changes can be handled by swipe gesture. It makes interaction functions

such as horizontal view change, switch back to the previous view, etc.

There is another method to take place of swipe gesture in vertical, which is using Digital Crown.

In case the vertical swipe interaction is not a subset of Digital Crown’s function. It is because of Digital

Crown cannot call notication center when user stay on watchOS dial interface.

2.2 Special Techniques

Apple Watch introduced three more interactive hardware based on traditional interactive touch

screen mode. They are Digital Crown, Force Touch and Haptic Engine, as shown in Figure 2.2

(a) Digital Crown (b) Force Touch (c) Haptic Engine

Figure 2.2: Special Techniques: Digital Crown, Force Touch and Haptic Engine is the most signicant

feature in Apple Watch than other smart watch products.

2.2.1 Digital Crown

The Digital Crown lets users scroll content without obstructing their view of that content. It

is a new interaction technique introduced by Apple Inc. This technique is compared with the two

revolutionary interaction techniques: Desktop Mouse and Multi-Touch screen when Apple Inc. were

introducing this, which means Apple Inc. maintain Digital Crown is a revolutionary technique on smart

watches.

This interaction take the advantage of traditional watch clock function, which is visible but also

insuciency. In a conventional watch, the clock dial in the normal state does not have any function,

only when the knob is pulled out and the only function is adjusting time. This device is uselessness

in the most of time. It’s a typical demand-driven design. Designers are not carefully think about its

existence but just habitual in use.

On the Apple Watch, content are displayed vertically and it’s restricted on a xed width. When

Digital Crown scroll up and down, the content will be change in vertical direction, as shown in Figure

2.2(a); Besides, Digital Crown can also adjust the playback volume of the music.

2.2.2 Force Touch

Apple Watch display not only senses a touch, it senses the force applied by user’s nger and

responds accordingly. Force Touch is the rst applying in consumer products. In academia, this

technology has been studied years.

[27–29] studied how to enhance touch feedback, force level and touch area. Force Touch applied

them into Apple Watch.

Image Source: https://developer.apple.com/watch/human-interface-guidelines/

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Force Touch contains two level of touch behavior, the rst level is traditional touch action, it

can reconize normal tap gesture; The second level is actually Force Touch, at this moment, user need

increase nger’s force value, then watchOS will response and handle this interaction. As illustrate

on Figure 2.2(b), obviously, this interaction is a typical interaction mode without aordance, these

functions are perceptible only the operation performed.

2.2.3 Haptic Engine

Haptic Engine is built-in vibration member, as illustrate in Figure 2.2(c). It is an important way

to get the user’s attention and to convey important information. Haptic Engine are not just a normal

vibration but also a high precision hardware, it can transfer user’s heartbeat to another, make user feel

Force Touch perform notication, then enhance the user experiences of Apple Watch. Thus, whatever

how to modify current interaction mode into alternative design, we should always provide the Haptic

Feedback to users. In other words, Haptic Engine vibration feedback is provided to be the key of

alternative interaction design.

2.3 Other Techniques

In addition, except the basic interaction techniques above, there are two highest priority interactive

method on Apple Watch, as Figure 2.3 illustrated

(a) Side Button (b) Voice Control

Figure 2.3: Other interaction techniques：Side button gives us the top-level interaction for Friends，

and Siri is the only text entry on the Apple Watch.

Two buttons on the side of Apple Watch actually is the highest priority of interactions, because

wherever the user in watchOS, it alway takes user to the Friend view and Siri interface (Note that a

single tap of Digital Crown is not able to mark as highest priority interaction, because a single tap

perform results may not be unique: It may be in the dial, or it may be in total App Interfaces).

2.3.1 Side Button

Apple Inc. may have noticed that the Apple Watch application gains its complexity and inconve-

nience, they follows the design of the power button on smart phone and they still add another physical

buttons called Side Button next to the Digital Crown, as Figure 2.3(a).

The existence of two dierent pieces of renovation interactive content inside the Side Button, when

user perform a button click, it will access the Friends screen. From this screen, they can call friends,

图片源：https://developer.apple.com/watch/human-interface-guidelines/

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

send message, or interact by sending sketches, taps, and even their heartbeat. When user press the

Side Button twice, it will access the Apple Pay.

2.3.2 Voice Control

Although there are a lot of input method design on wearable devices, but Apple Watch did not

take any of them on watchOS, text input entrance (even virtual keyboard) are not directly setup into

watchOS, thus Siri becomes the only interface allows user input text by speaking, as shown in Figure

2.3(b).

For weak up Siri, user have two way to do. One is say ”Hey Siri” when Apple Watch is on awake

state; Another is give a long-press to Digital Crown.

However, no matter which way to choose, text input results will not ideal. This is because Siri’s

nature language recognition process based on statistics and it always require a paired iPhone to access

Internet. Any part of this gets a problem can crash the whole input progress.

It is worth mentioning that the interaction method we discussed above, only Siri voice control is

a contact-free interaction design.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

3 Alternative Design of Contact-free

We analyzed the interaction mode on Apple Watch in last chapter, all the interaction method

required two hands to be performed, except the voice interaction. Considering the actual scenario, if a

user’s hands are free to do anything, there is no obvious reason to make things done on smart watches

due to we have a powerful smart phone, moreover, to operate a phone actually only needs one hand.

Therefore, this contact type of interaction on smart watch essentially isn’t a stickiness interaction mode.

3.1 Interaction Pattern Design

3.1.1 Tap & View Switch

Tap and view switching operation involving the following ve operations: ordinary tap, Force

Touch tap, switch to left view, switch to right view, switch to previous view.

We design the alternative interaction of tap gesture to be two nger pinch, thumb pinch with the

other four ngers can fully mapping all tap interaction, such as thumb with index nger can replace

normal tap event, thumb with middle and ring nger can replace view switching, and thumb with little

nger can replace back to previous view.

Besides, pinch gesture with thumb and index nger can also simulate Force Touch, see more details

in Section 3.1.3.

3.1.2 Swipe & Continuous Adjustment

Swipe and continue changes actually belongs to Digital Crown interaction in dierent scenarios.

In General view down swipe scenario, it can scroll view contents up and down; when the interactive

object choose as slider bar, it also can adjust value continuously.

We can implement this interaction by using two ngers pinch gesture on single hand, which means

thumb slides on index nger, and if the interactive object is slide bar, then this kind of two ngers

swipe can also regulate slide bar value.

3.1.3 Force Touch Simulation

An open source project called Forcify [30] aims to develop an universe touch events handler frame-

work, convert any Web App click events to 3D Touch Events

. But the time handler need to customize

by developer and the force touch simulator is a linear function. Therefore, this project needs to be

improved.

At rst, we split touch events to two stage, the rst stage is as normal touch events, the second

stage is as Force Touch Events, and set content DELAY to express the time delay of Force Touch, and

DURATION to express the time duration of Force Touch from minimum to maximum. Now we focus

on this two constants value.

In AugmentedTouch Project [31], we applied a user study and published a dataset contains 16

users of 4 dierent posture, altogether contains 61440 times tap data. In this dataset, we record the

time of user’s nger stay on screen and Figure 3.1 shows its distribution.

Apple rename Force Touch in iOS as 3D Touch.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Duration of Finger Stay on Screen(ms)

0~50

50~100

100~150

150~200

200~250

250~300

300~350

350~400

400~450

450~500

500~550

550~600

600~650

650~700

700~750

750~800

800~850

850~900

900~950

950~1000

Count

1000

2000

3000

4000

5000

6000

7000

8000

Count

Log-Normal: µ =1.9626,σ=0.54024

Maximum Likelihood:x=260.4776

Figure 3.1: Statistics of nger stay on screen

As it shows the distribution on Figure 3.1 we can get the time of a nger stay on a screen mainly

focus on 260ms, obey to log-normal distribution. In this regard, we could assume the average time of

nger stay on touch screen is 250 ms, so DELAY equals 250 ms; In addition, we set the total duration

is ve times of the ordinary touch events time, so DURATION equals 1000 ms; Then to trigger the

Force Touch user need stay nger on touch screen longer than 1.25 seconds. Considering human’s nger

pressure of touch, we also assume the intensity of the rate of force value is linear, then we can ensure

the entire curve is a smooth curve.

In conclusion, let the time of pressing to be t

press

, then Force Touch can be simulate by a formula

3.1.











0 if t

press

< DELAY



press

−DELAY

DURATION



if 0 < t

press

− DELAY < DURATION

1 Otherwise

(3.1)

Among the formula, v

represents the pressure value, DELAY = 250, DURATION = 1000, both

in milliseconds(ms), as shown in Figure 3.2.

press

(ms)

DELAY

DELAY+DURATION

Figure 3.2: Force Touch functional image

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

3.1.4 Haptic Feedback Design

According to the developer documentation of watchOS

, Haptic Engine can express several types,

as its illustrate on Code 3.1.

1 public enum WKHapticType : Int {

2 case Notification

3 case DirectionUp

4 case DirectionDown

5 case Success

6 case Failure

7 case Retry

8 case Start

9 case Stop

10 case Click

11 }

Code 3.1: Types of Haptic feedback

Success, Failure, Retry type of Haptic feedback are native results in watch operation. Then we

set Success, Failure as same as its original function, but the context of Retry is basically as same as

Failure, so we keep this feedback for the future.

Notication feedback still needs to present notication feedback, so we don’t change its design;

DirectionUp and DirectionDown only happens on Digital Crown scroll up and down in the end, so we

follow this function to two ngers swipe, when value comes to the maximum value, use DirectionUp

feedback and vice versa.

Start, Stop, Click and Retry this four type of feedback is suitable for thumb with index, middle,

ring nger pinch (three) and Force Touch operate, which means feedback of Click is for normal tap,

feedback of Stop is for Force Touch, feedback of Start is for view switching to next, and feedback of

Retry just right for view switching to previous.

3.2 Completeness of Alternative Design

Figure 3.3: Summary of Contact-free Interaction

watchOS version stay on 2.2 when this thesis nished.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

In this chapter, we fully redesigned the interaction mode on Apple Watch, it simplies the in-

teraction logic between elements, as a summary we list the design logic in Figure 3.3. This design,

combine with the Haptic feedback, convert the basic tap, swipe, Digital Crown, Force Touch and etc.

native watch interaction to a contact-free gesture interaction. It simplied ten native interaction with

two-step logic to only eight alternative interaction with single-step logic, eliminated contact-required

interaction.

As illustrate in Figure 3.3, this interaction scheme is completeness, which means we archived all

the function without original interaction method.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

4 System Design

4.1 Framework Selection

4.1.1 LeapMotion & LeapJS

LeapMotion [32] is a general term for a hardware input device and its software, its hardware built

by two depth camera and an infrared detector that can detect the hand within the eld of view; its

software built-in a skeleton model of hands for hand reconstruct

LeapMotion SKD provides two varieties of API for getting the LeapMotion data: a native interface

and a WebSocket interface, WebSocket provides JavaScript interface in browser environment. It is

conforms to RFC6455

, and running on desktop default port 6437.

LeapJS is the JavaScript framework for LeapMotion Controller. Using LeapJS enables Web font-

end communication with LeapMotion, and this framework is used for processing LeapMotion JSON

data. With NodeJS, LeapJS also can running on server side, thus we can also processing LeapMotion

interaction on server side by JavaScript. Leap API gives a Frame object, which contains a Hand object

if the hand of users can be detected in LeapMotion eld of view. LeapMotion through its built-in model

of hands, reconstruct hands object and then provide its to developers. for example, it provides hand

direction, hand nger direction, hand position in LeapMotion coordinate system. With LeapMotion,

developer and researcher can avoid the basic image segment works and other stu, them keep focus on

gesture algorithm [35–39].

In addition, 3D gesture already gains its application in simple method, [40] gives a LeapMotion

control interaction for TV.

4.1.2 watchOS & WatchConnectivity

watchOS is the operating system that runs on the Apple Watch. When watchOS just released,

applications only as an extension for iOS App, watchOS only illustrate animations, all code runs on

iOS.

With lunching of watchOS 2, now applications can use WatchConnectivity framework to pass data

between iOS and watchOS.

WatchConnectivity framework aims to makes message between iOS and watchOS to be a message

in system level, developer shouldn’t care much about it. These communication will start a session

and only WCSession.defaultSession.reachable is true, the delivery can be started. So if we would use

WatchConnectivity, we must implement WCSessionDelegate protocol.

Besides, WatchConnectivity contains two dierent communication module: background module

and interactive module. In background module, OS will push the message into a queue, then process it

when App wake up; In interactive module, message must transfer through OS and it is an immediately

communication, only happens when WCSession.defaultSession().reachable is true.

It is worth mentioning that, in interactive module, even if the iOS client program has not yet

started, the Apple Watch is still able to wake up an iOS App from background.

LeapMotion’s ocial accuracy is 0.01 mm, but [33, 34] report its actual accuracy is 0.2 mm

http://tools.ietf.org/html/rfc6455

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

4.2 Architecture

4.2.1 Communication Structure

watchOS was split from iOS App Extension since 2.0 version, since then, watchOS can actually

execute code on Apple Watch, that make watchOS App can management communication [41] between

iOS App and itself, the structure as illustrate on Figure 4.1. This makes timely communication with

the outside world be possible.

watchOS

Watch App

WatchKit

extension

iOS

iOS App

Figure 4.1: Connection between Watch App, WatchKit Extension and iOS App

However, there are still load of limits for networking on watchOS. In watchOS 2, Apple Watch

only when its paired iPhone disconnected and its also in a saved WiFi environment that can access

internet by using NSURLSession, these conditions are rigor.

In consideration of above, we designed the communication architecture and illustrate it on Figure

4.2.

Server

Interaction Handle

watchOS AppiOS App

Leap WebSocket

Request Handle

Figure 4.2: Communication Architecture: iOS App as a router to transfer message between

watchOS and server, instead of watchOS directly communicate to server

Among this architecture, iOS App will be the bridge for watchOS App and server for processing

interaction message, watchOS only cares content present.

4.2.2 Client Structure

Client side includes iOS side and watchOS side, as shows on Figure 4.3.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

iOS App

Communication Layer

Networking Layer

Message

Protocol Handle

WatchConnectivity

watchOS App

Communication Layer

View Control Layer

WatchConnectivity

InteractionPresent

Figure 4.3: Client side structure: iOS App process server message then send the results to commu-

nication layer to watchOS, watchOS react the interaction manipulate UI elements by view controller

layer when it recived the message.

4.2.3 Server Structure

Server side structure as its shown on Figure 4.4, the core module is Interaction Handle Layer, this

layer process raw interaction data from sensors and post the interaction results to Request Handle layer

then distribute it to suitable devices.

Server

Interaction Handle

Request Handle

Message

Protocol

Handle

Commons

MLiteKitLeapJS

……

Figure 4.4: Server side structure: Interaction Handle Layer process raw interaction data, then post

it to Request Handle Layer for data transfer

4.3 Communication Protocol

We need design the interaction message communication protocol between watchOS, iOS and server.

According to Figure 3.3, basic interaction gesture contains grab, two ngers swipe and tree dierent

tap(thumb with index, middle, ring nger), the two ngers means thumb and index nger.

For this reason, interaction message eld design as in Table 4.1.

Table 4.1: Interaction Protocol Fields

Field Type Description

pinchTimeStramp Date Time of pinch operate

pinchIndex int Finger index of pinch, from 1 to 4 express index nger to little nger, -1 means no nger

pinchStrength double Strength of pinch, from 0 to 1, real number

grabStrength double Grab strength, from 0 to 1, real number

forceValue double Simulate value of Force Touch

4.4 Demonstrate Programs

Although we conducted a comprehensive interactive design, but due to the restriction on the

development of Apple Watch, we are unable to operate the view in system-level. In this thesis, we

gives ve alternative interactive demos with dierent eects.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

• Demo 1: This demo shows the alternative interaction design of tap gesture;

• Demo 2: This demo shows the alternative interaction design of swipe gesture;

• Demo 3: This demo shows the alternative interaction design of Digital Crown;

• Demo 4: This demo shows the alternative interaction design of Force Touch;

• Demo 5: This demo illustrate a contact-free game experiences on Apple Watch.

Above ve demos video can be found in YouTube

, for other related resources, such as source

code, see Appendix A.

https://www.youtube.com/playlist?list=PLwUqqMt5en7c2QaQ_DkuvZm9dGTz6RjRM

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

5 Implementations

5.1 Development Environment

Whatever client side or server side, there always framework depended, so build development envi-

ronment is inevitable. In this project, the server is encoded using NodeJS. With limited space, for basic

NodeJS environment, such as common conguration tool, NPM package management are omitted in

this thesis. Here we introduce the most important framework structure and build of this platform,

LeapJS; On client side, we use Swift to code and we do not import any third party framework, but

WatchConnectivity need to be use for communication [42]. After iOS 9, every App in iOS need to use

HTTPS request, but there is a way to congure HTTP request in iOS 9.

5.1.1 Conguration of Local LeapMotion

LeapMotion provides development environment in Mac OS X, and provides a variety of program-

ming language. Based on above discussion, we need to congure the server LeapMotion body and

LeapJS.

1. Installation

First of all, to use LeapMotion API, we have to import LeapSDK into our desktop host application.

Secondly, we need add dependency of LeapJS in Node App’s package.json le:



"dependencies ": {

"leapjs ": "^0.6.4"

}



 

The next step is install LeapJS by npm install.

2. Congs

By the limitations of LeapMotion [32], WebSocket service doesn’t open a non-local access in default

setting, so we will need to congure Leap then enable non-local client connections.

This requires to congure LeapMotion prole. Before we modify the conguration, we have to close

LeapMotion related services. On Mac OS X, we use the following command to shut down LeapMotion

daemon:



sudo launchctl unload /Library/LaunchDaemons /com .leapmotion .leapd .plist



 

Next, we need to modify LeapMotion prole. According to the ocial documents , Leap contains

two dierent congurations and the highest priority belongs to control panel, so we need edit the

following directory:



$HOME/Library /Application\ Support/Leap \ Motion



 

Find cong.json conguration le, editing cong.json le and add an arbitrary position in the

conguration eld:



"websockets_allow_remote ": true



 

nally obtained:



"configuration ": {

"websockets_allow_remote ": true ,

"background_app_mode ": 2,

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

"images_mode ": 2,

"interaction_box_auto ": true ,

"power_saving_adapter ": true ,

"robust_mode_enabled ": false ,

"tracking_tool_enabled": true

}



 

save and quit, restart LeapMotion Service and then we nished the conguration of LeapMotion:



sudo launchctl load /Library/LaunchDaemons /com .leapmotion .leapd .plist



 

5.1.2 watchOS Networking Access

From iOS 9. Apple introduced App Transport Security feature for iOS App, this causes iOS App

could not access to the Interaction with HTTP requests.

Due to watchOS networking need iOS App to transfer its requests in the most of time, we need

applying two more conguration steps, and the results shows on Figure 5.1.

1. Add NSAppTransportSecurity as a Dictionary type into Info.plist;

2. Add NSAllowsArbitraryLoads key as a Boolean type into NSAppTransportSecurity, then set its

value YES.

Figure 5.1: Conguration results of info.plist le

5.2 Server Side

To archive a HTTP Server with port 10086 is the rst step of developing, it is implement by

NodeJS and codes shows on Code 5.1.

1 var http = require ('http ');

2 f u n ction handler (req , res) {

3 res. writeHead (200) ;

4 }

5 http .createServer(handler ).listen (10086);

6 console.log(" Server running at http :// locoalhost :10086")

Code 5.1: Create HTTP server on 10086

Now, we focus on how to process LeapMotion Gesture in the following content.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

5.2.1 Tap and Force Touch Processing

LeapMotion Software itself will reconstruct hand model of users, and its APIs provide the skeleton

data, including ngers tip position. To recognize pinch gesture, we can calculate two nger tips, so

that we can iterate all nger tips except thumb, the implementation code shows on Code 5.2.

1 f u n ction findPinchingFinger (hand , closest ){

2 var pincher;

3 //var closest = 500;

4 for( var f = 1; f < 5; f++)

5 {

6 current = hand.fingers [f];

7 distance = leap.vec3.distance(hand.thumb .tipPosition , current .tipPosition);

8 if( current != hand. thumb && distance < closest )

9 {

10 closest = distance ;

11 pincher = current ;

12 }

13 }

14 // mark time of start pinch

15 if (pincher .type != pf. pinchIndex ) {

16 // pf is a gloable value for message protocol

17 pf.pinchTimeStramp = Date.now ();

18 }

19 ret urn pincher ;

20 }

Code 5.2: Tap Gesture Recognize

We have discussed Force Touch simulation in Subsection 3.1.3, now the implementation of JavaScript

code shows on Code 5.3.

1 f u n ction forceValue(){

2 // pf is the global message protocol object

3 if (pf. pinchTimeStramp === 0) {

4 re turn -1;

5 }

7 var delay = 250;

8 var duration = 1000;

10 // calculate force touch value

11 var value = ((Date .now() - pf.pinchTimeStramp ) - delay )/duration ;

12 var force = (value >=1 ) ? 1 : value*value ;

14 ret urn force;

15 }

Code 5.3: Force Touch Simulation

5.2.2 Two Fingers Swipe

For two ngers swipe, it seems dicult to identify, but in fact we can converted it to pinch

degree between two ngers. In fact LeapMotion itself provides thumb and index nger pinch degree

parameter, but because of its results is universe for every ngers and cannot customize by developer.

For this reason, we have to reimplement this function.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Thumb

Finger

Vector

Index

Finger

Vector

Palm

Normal

Vector

(a) Before

Thumb

Finger

Vector

Index

Finger

Vector

Palm

Normal

Vector

(b) After

Figure 5.2: Two nger swipe: two nger swipe can trigger events by palm normal vector and nger

direction vector, 3D models in this gure are generate by LeapMotion Visualizer

Two nger swipe gesture shows on Figure 5.2, when hand presented as Figure 5.2(a), the normal

vector of palm and direction vector of index infer are parallel, thumb direction vector and palm normal

vector are almost vertical, so we can use this character to trigger this events, as its illustrate on Code

5.4.

1 f u n ction findPinchingStrength (hand ) {

2 var thumbDirection = hand.fingers [0]. direction ;

3 var indexDirection = hand.fingers [1]. direction ;

4 var dotProduct = leap .vec3 .dot( thumbDirection , indexDirection );

5 // remove grab condition

6 if (dotProduct < 0.7 && hand .grabStrength < 0.85) {

7 r e turn hand.pinchStrength ;

8 } else {

9 r e turn -1;

10 }

11 }

Code 5.4: Two ngers swipe recognize

5.2.3 Grab

We don’t need reimplement grab gesture, LeapMotion itself provides grab parameter of a hand,

we can straightly get this data from Frame object, as shown on Code 5.5.

1 grabStrength = functio n getGrabStength (frame) {

2 ret urn frame.hands [0]. grabStrength ;

3 }

Code 5.5: grab recognize

5.3 Client Side

The key of client encoding is the communication layer and view controller layer , communication

layer also contains communications between iOS, watchOS and server, we use Swift

to complete the

associated code [43, 44].

Swift version 2.2.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

5.3.1 View Controller Layer

View Controller Layer is basically a response for interactive message, as an example, we shows the

core code of design for two nger swipe gesture as a Digital Crown alternative design.

Picker class on Apple Watch need developer load resources themselves, code as shown on Code

5.6.

1 var images : [ UIImage ]! = []

2 var pickerItems : [ WKPickerItem ]! = []

3 for (var i=0; i <= MAX_NUM; i += 1) {

4 let name = "progress -\(i)"

5 images.append (UIImage (named: name)!)

7 let pickerItem = WKPickerItem ()

8 pickerItem .title = "\(i)"

9 pickerItems .append (pickerItem)

10 }

11 let circleImages = UIImage .animatedImageWithImages (images , duration: 0.0)

12 circleGroup .setBackgroundImage (circleImages)

13 picker .setCoordinatedAnimations([ circleGroup ])

14 picker .setItems(pickerItems)

Code 5.6: Load resources of Picker class

We mapping the pinchStrength value from [0,1] to [0, MAX_NUM], then use setter function of

Picker to modify UI interface, as shown on Code 5.7.

1 func indexSelect (message: InteractionMessage ?) {

2 index = Int( message !. pinchStrength * MAX_NUM )

3 // haptic feedback

4 watchDevice . playHaptic(WKHapticType.DirectionUp)

5 picker.setSelectedItemIndex (index)

6 }

Code 5.7: Picker UI manipulate

5.3.2 Communication Layer

To communicate with the server, we can use HTTP request from server to gesture data, which is in

JSON format. In Code 5.8, completeFlag guarantees only after a previous request completes program

will trigger next request out. The constructor of Gesture class will parsing JSON data and contract a

Gesture object model from the data dictionary.

1 func connectServer () {

2 // member variable

3 if completeFlag == 0 {

4 return

5 }

6 task = session.dataTaskWithURL (url , completionHandler : { (data , res , error ) ->

Void in

7 if let e = error {

8 print("dataTaskWithURL fail: \(e. debugDescription )")

9 return

10 }

11 if let d = data {

12 print("\( NSString (data : d, encoding : NSUTF8StringEncoding))")

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

14 if let jsonObj = try? NSJSONSerialization . JSONObjectWithData (d, options :

NSJSONReadingOptions . AllowFragments ) as? NSDictionary {

15 self.Gesture = Gesture (fromDictionary : jsonObj!)

16 }

17 self .completeFlag = 1

19 }

20 })

21 task!. resume()

22 }

Code 5.8: Communication between iOS and Server

To establish a connection between iOS and watchOS, the only way to archive this is to use Watch-

Connectivity framework, Code 5.9 and Code 5.10 list the core implementation of send message to

watchOS and receive message from iOS.

1 func updateMessage () {

2 if WCSession.defaultSession ().reachable {

3 let content:[ String:String] = ["x": PinchFinger .text !, "y": PinchStrength.text!,

"z": GrabStrength .text !]

4 let message = ["up": content]

5 WCSession .defaultSession ().sendMessage(

6 message , replyHandler : { ( replyMessage) -> Void in

7 print("send success ..")

8 }) { (error ) -> Void in

9 print(error)

10 }

11 }

12 }

Code 5.9: Communication between iOS and watchOS: Send Message

1 extension InterfaceController : WCSessionDelegate {

2 func session(session: WCSession , didReceiveMessage message : [ String : AnyObject ])

{

3 guard message ["up"] as? [String:String] != nil else {return }

4 let contents = message["up"] as! [String : String ]

5 self .interaction(contents)

6 }

7 }

Code 5.10: Communication between watchOS and iOS: Receive message

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

6 User Study

We conduct an user study above alternative design from April 14, 2016 to April 15, 2016, which is

located at the Southwest University for Nationalities Wuhou Campus Bibliothek ground oor reading

room.

6.1 User Tasks

In this study, there are ve operations we need to evaluate: tap, force touch, digital crown, swipe

left and swipe right. According this, we designed two completely group tasks, and they are Calling(two

way to archive) switch watch digital plate. These tasks are above ve operations combinations.

• Task 1: Use side button. Find top contacts, then select a person and make the call. The task

logic is rise hand rst, push the side button and then rotate Digital Crown to select a person, tap

to select this person and make the phone call.

• Task 2: Use Call App. Push Digital Crown enter into Application desktop rst, tap Call App

then make the phone call. The task logic is rise hand rst, click Digital Crown, tap screen to

select Call App, select a person and then tap to make a phone call.

• Task 3: The task of switch watch layout is simple than the other two tasks. The most important

operation is make a Force Touch on watch screen. The task logic is rise hand rst, make a Force

Touch on watch screen, and swipe to left or right change its interface, then tap the screen and

nish this task.

Before starting user tasks, we teach our participants about how to interact on a smart watch, and

teach them 13 gestures without any hint of alternative design, gestures are list on Table 6.1. We start

the user study until we make sure they can interact with Apple Watch well and also remembered these

13 gestures.

Table 6.1: Gestures Index

Index Description

1 Thumb swipe on index nger

2 Thumb swipe on middle nger

3 Thumb swipe on ring nger

4 Thumb swipe on little nger

5 Thumb pinch with index nger

6 Thumb force pinch with index nger

7 Thumb pinch with middle nger

8 Thumb force pinch with middle nger

9 Thumb pinch with ring nger

10 Thumb force pinch with ring nger

11 Thumb pinch with little nger

12 Thumb force pinch with little nger

13 Five ngers grab

After above three tasks, we show the rst part of questionnaire to participants, an requires par-

ticipants recall previous learning of gesture and related tasks to complete this part of questionnaire.

Then we goes to next stage of user tasks for alternative interaction:

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

• Alternative Task 1: raise hand, call Friends view (grab), choose contact (pinch), make phone

call (pinch 3 times);

• Alternative Task 2: raise hand, back to App homepage(grab 2 times), select phone App and

make the phone call (pinch 3 times);

• Alternative Task 3: raise hand, Force Touch (force pinch, long time), swipe to left or right

(thumb and middle nger pinch), tap to end selection (pinch).

Alter these alternative interaction task, we show the rst part of survey again, requires participants

modify their answer until the answer satisfy their own. When this nished, we show the second part

of questionnaire, see Appendix C, then we logout this user study.

6.2 Participants

Sample size of participants in a usability test sometimes do not have a xed program, the classic

theory is that usability test for serious usability problems can only take ve participants [45]. However,

[46] analysed increase sample size to 20 can nd more than 95% usability problems.

Hwang et al. [47] discussed the changing rule of sample size in usability test. R. Maceeld et

al. [48] gives the inuence of sample size in each study group, they point out that 5 to 10 is the best

base number in each study group. [49] suggested use sample ratio to evaluate sample size when the

number of evaluate task is inequal to user tasks per user.

In [50], they discussed how eye tracking inuence the usability test. The above discussion of its

literature, we conclude that given an algorithm 1 as usability testing sample size calculation method.

Wherein calculating the number of samples is determined by the following parameters:

1. group: number of user group(such as expert or freshman), this number should not large than 5;

2. task: Is the user task same between dierent group?

3. baseline: Will user study result inuence product decision or not?

4. design: evaluating design numbers in a user study, this number should not large than 5;

5. peruser: evaluating design number for each user, each user should not evaluate more than 5

design;

6. eyetracking: Is this user study enable eye tracking? If so, is it qualitatively or quantitatively?

Then we have the algorithm of participants sample size in usability test:

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Algorithm 1 Calculate minimum participant sample size

Require: group, task, design, peruser, eyetracking

Ensure: sample

base ← 10

sample ← 10

if eyetracking is qualitatively then

base ← 12

else if eyetracking is quantitatively then

base ← 39

end if

if baseline is yes then

base ← 16

end if

if design ≤ peruser then

ratio ← 1

else

ratio ← ⌊

design

peruser

⌋

end if

if task is same then

sample = ⌈(base + 6 · (gr oup − 1)) · ratio⌉

else

sample = ⌈base · group · ratio⌉

end if

In our user study, we randomly invite participants perform same tasks, and the results inuences

future design, so the parameters are group = 1, baseline = yes, task = same, design = 5, peruser =

5, eyetracking = no. According to algorithm 1, the sample size should not smaller than 16. We put

this user study information in social networks and also randomly invite people who get in or out from

library become a participant, and nally we have 21 participants.

6.3 Analysis and Results

6.3.1 Statistics of Participants

Out of 21 participants, 11 were male and 10 were female. The range of age was 18 to 28. Besides,

there are three participants have smart watch use experiences.

Participants who have use watch in daily life, all own more than two watches, and there are 9

participants consider to buy a new smart watch. In watch users, only 6 participants plan to buy a

smart watch and most of them regard its prices.

We have 26.57% (6 people) participants neither do not have a watch or do not consider to buy it.

In another words, they will never buy a smart watch if smart watches is not a necessary product.

6.3.2 Social Acceptable

In the rst part of questionnaire, we set up a social acceptability problem, it is ”how to treat

strangers raise hand excuse gestures in Table 6.1?”. In these participants, there are 42.85% (8 people)

participants thought it was exercise, 14.28% (3 people) thought it was thinking, and 33.33% (7 peo-

ple) thought it was a performance art, 19.04% (4 people) didn’t care about it and 4.76% (1 person)

considered ”insane”.

From the above results, all participants accept this action except 4.76% (1 person).

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

6.3.3 Comfortable Rate of Gestures

Every participant makes a gestures comfort voting, gestures is in Table 6.1, results shown on

Figure 6.1.

Gesture Index

1 2 3 4 5 6 7 8 9 10 11 12 13

Vote Count

Figure 6.1: Vote results of gestures in Table 6.1

The results showed that more than 16 people (76.19%) vote three gestures, they are swipe between

thumb and index nger, pinch with thumb and index nger and ve nger grab, they are in the rst

echelon. Thus these three single hand gestures has comfortable and easy characteristics for users.

Next, the second echelon, more than 11 people (52.38%) vote to thumb swipe on middle nger,

long pinch with thumb and index nger. It is indicative that these gestures’ comfort lower than the

rst echelon gesture.

It is worth to mention that in all 21 participants, no one choose thumb pinch with little nger,

which means this gesture is the most uncomfortable gesture among them. In contrast, we can use this

gesture to operate some dangerous operation, such as delete.

6.3.4 Gesture Intuition

We are also studied user’s gesture intuition, to verify the reasonableness of the alternative design

in the rst part of questionnaire. Voting results as shown in Figure 6.2, in which the red color indicates

more deeper gets gesture intuition for user more higher.

Figure 6.2: Heatmap of intuition vote results

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

According to the results, participants thought the tap and force touch gestures is a strong intuitive

operation. In tap gesture, participants has two obvious point of view, some may say thumb swipe on

index nger is more intuition, and some may say quick pinch is more logical.

Especially for Force Touch, the user does not exist distinct operation divisions. participants were

generally more inclined to use long pinch. Digital Crown’s gesture were more modest, but it can be

observed that user tend to use thumb swipe on index nger.

However, for swipe gesture, participant becomes confused, there are no obvious results shows. So

we still need to considering this part of gestures carefully, and sometimes we may need user to learn

these gestures.

6.4 Conclusions

We can conclude, through this user study, the alternative design of tap, Force Touch and Digital

Crown is more comfortable and intuitive for user’s operation. As it happens, this is the most frequency

operation for smart watch interaction.

For others, there is no obvious logic and intuition for interaction. Fortunately, the Apple Watch

display content vertically so swipe left and right are not frequency, back to previous view belongs to a

sensitive operation, we could use these gesture to improve user vigilance and pay more attention on it.

In consequences, our alternative interaction design is appropriate for most normal users.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

7 Future Works

7.1 Weakness

We have discussed few unbreakable watchOS and LeapMotion software-related problem, due to the

limits of performance and energy on watchOS, the communication of interaction message processing

is the most important part of coding. The next contents we focus on problems of hardware and the

interaction design itself.

7.1.1 Hardware Related

As a prototype, in this thesis we use LeapMotion to process and recognize hand gesture, unfortu-

nately, LeapMotion required a USB connected desktop computer, which is totally restricted the scope

of interact scenes, it cann’t used in other scenes.

In [51], Gilles et al. put a deep camera on the top of shoes and create a area above it. Likewise,

we can design a mobile LeapMotion solution, as illustrate on Figure 7.1.

Batterry

LeapMotion

Raspberry PI

Wi-Fi Module

Figure 7.1: Hardware Structure: The hardware contains a 5V-2A battery, a Raspberry Pi B+ with

WiFi Module, a LeapMotion Device.

In this scenario, the mobile charger supply output 5V-2A for driving Raspberry Pi; Raspberry Pi

connect server for interaction message communication by WiFi-Module hardware; then LeapMotion

connect to Raspberry Pi by USB. Until now, the design could be completely mobile.

Unfortunately, LeapMotion’s host OS requires at least 2GB RAM and the SKD only supported

on Ubuntu Linux, Mac OS X and desktop Windows. Currently the second-generation Raspberry Pi

B+ only have 512M RAM. But we expect to see the future of Windows 10 could running on raspberry

PI when RAM expand to 2GB.

7.1.2 Interaction Related

Alternative design presented in this thesis although completely archived original interaction design

and also user study results reect it is advisable, but once the user tasks become an complexity task,

the eciency of alternative design will poor than original. Considering Keystroke-Level Model [52], we

present a Watch-KLM, it’s basic task step shown in Table 7.1.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Table 7.1: User task basic steps

Value Description

K Tap

F Force Touch

S Swipe

T Target Select

R System Response

M User Thinking

H Highest Priority Operate

For a normal tap gesture, we assume that in these two design the execute time of pinch gesture are

same. Note that in alternative design tap an object need to select an specic object, thus K

contact-free

contact

; For Force Touch, alternative interaction can directly measure user’s press value, however in

section 3.1.3 we gives two stage of Force Touch simulation longer than original design, that cause

contact-free

> F

contact

For T 、M and H, if user has already familiar with these two design, we may consider that time

of execute exactly the same. For T , alternative design need iterate all objects in a view, apparently

we have T

contact-free

> T

contact

. In addition, alternative design spend time on communications, so

contact-free

> R

contact

Thus, contact interaction and contact-free interaction can only be a linear combination of these

seven basic steps. For a same task,the coecients are same. Based on above analysis, it is clearly that

we have the formula 7.1.

K,F,S,T,R,M,H

Contact Task < Σ

K,F,S,T,R,M,H

Contact-free Task (7.1)

Left part in formula 7.1 will signicantly less than the right part when user tasks become much

complicated.

7.2 Improvements

At last, we discuss how to improve the future design of the system architecture in this application,

as well as over all system scalability.

7.2.1 Recognition Approach

We use LeapMotion as a gesture recognize solution for alternative interaction, but in reality Leap-

Motion is a desktop interaction hardware, a strong dependence vision method based solution is not

quite suitable for a portable and wearable devices. Here we discuss another hardware for detection.

Myo [53] is a xed machine on human arm. It detect the electronic signal of muscle, see Figure

7.2

. Myo use built-in WiFi module to deliver the signal. Myo also opened its recognize protocol

enables developer can specify its gesture system.

Image source: https://www.myo.com

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Figure 7.2: Myo Device: Myo install on user’s arm, it will detect the electronic signal of muscle.

The morphological of Myo is completely suitable for ergonomically design, this because of the band

of watch is the best form of existence for Myo. When Myo can be reduced and designed into watch

strap, the recognition will no longer dependent on individual external devices, it is more ecient to

complete interaction and data processing.

7.2.2 IPC

In section 4.2.1, we discussed the communications architecture’s design, this architecture can also

be applied in general cases.

Figure 7.3 is the expand from the previous design, we introduce a Interaction Perception Cen-

ter(IPC, which is a cross platform interaction communication solution for desktop and mobile devices

we presented) service [54]. We put IPC as a system level model for interaction, its Sensor Handle Pro-

tocol (SH Protocol) can dismiss, formalize hardware dierences, and Cross-Device Interaction Protocol

(CDI Protocol) enables user devices communicate with IPC service.

Server

IPC Core Service

User

Devices

Mobile Phone

Sensors

Cross-Device Interaction Protocol

Hosting OS

Raw Processing Layer

Sensor

Message

Protocol

IPC Local Service

Figure 7.3: Universal communication architecture: Introducing IPC service on server and client

side.

Dierent sensors with dierent diver, but if there exist a SH protocol and the data can be processed

by host OS, then OS will send these message to sever together for a second process, server side distribute

the interaction message to any other devices through CDI protocol; Besides, if IPC core service is not

available, the sensor host OS can still communicate with client for some simple interaction oine.

IPC was designed as a pure software solution in the beginning, it’s purpose to give a distributed

data communication and analysis platform between mobile devices, desktop computers and sensors,

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

now we conclude few implicit from above discuss: Firstly, design of universal interaction system should

not transport to IPC server directly, data from sensors should machining on its host OS at rst, then

the process data should be packaged through a message protocol to guarantee all sensor data can access

IPC interface; Secondly, IPC interaction message must be optional for users, these interaction in any

cases should provides a primary interaction at least. The scilicet marks passive interaction should be

implicit and alternative.

7.3 Closing

In this thesis, we combine with the Haptic feedback, convert the basic tap, swipe, Digital Crown,

Force Touch and etc. native watch interaction to a contact-free gesture interaction. It simplied

ten native interaction with two-step logic to only eight alternative interaction with single-step logic,

eliminated contact-required interaction method, solved the bimanual interaction within smart watches,

and explained the completeness of this alternative design.

With this work, we overcome the hardware platform and software interface limitations, developed

an architecture in the progress and its us a set of implicit for build an interactive system.

Through user study, we evaluate the whole alternative design and gesture comfortable rate and

intuitive rate. These alternative designs and the result shows the interaction operate logic and gesture

comfortable all acceptable.

Overall, the innovative human-computer interaction is often accompanied by a combination of

hardware and software, in the process of software layer sometimes subject various limitations of hard-

ware platform. A set of open architecture design of hardware and software not only from hardware

to software vertical integration, but also the need to do a horizontal products usability design, even

considerate on users. These contents oat above the water surface, it’s only the tip of the iceberg.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Appendix A Licence

The program, paper and etc. in this thesis are open source, they can be found in:

• Thesis homepage: https://changkun.de/blog/bechelorthesis/;

• GitHub repostory: https://github.com/changkun/BachelorThesis/.

In case, all related text, picture and video content are licensed under a Creative Commons

Attribution-NonCommercial-ShareAlike 4.0 International License

The other part in this thesis (program source code) is licensed under a GNU Public License Version

3.0.

http://creativecommons.org/licenses/by-nc-sa/4.0/

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Appendix B Questionnaire - Part A

Welcome! :)

Thank you for join our study! This questionnaire is the rst part of out research, we need you to

ll this simple questionnaire. Quesions marked by * is required

1. Please select the most comfortable options * At least ve options

• Thumb swipe on index nger

• Thumb swipe on middle nger

• Thumb swipe on ring nger

• Thumb swipe on little nger

• Thumb pinch with index nger

• Thumb force pinch with index nger

• Thumb pinch with middle nger

• Thumb force pinch with middle nger

• Thumb pinch with ring nger

• Thumb force pinch with ring nger

• Thumb pinch with little nger

• Thumb force pinch with little nger

• Five ngers grab

2. What would you think about strangers raising hand make some gesture which are

showed on question 1?

Please write your idea

3. Please connect the left side and right side operate options if you think they are

suitable *

Options on the left side can only connect to only one option on the right side, you can pass a

connection if they are not suitable.

Thumb swipe on index nger

Thumb swipe on middle nger

Thumb swipe on ring nger

Thumb swipe on little nger

Tap gesture Thumb pinch with index nger

Finger force touch a screen Thumb force pinch with index nger

Tune knob Thumb pinch with middle nger

Finger swipe left on a plane Thumb force pinch with middle nger

Finger swipe right on a plane Thumb pinch with ring nger

Thumb force pinch with ring nger

Thumb pinch with little nger

Thumb force pinch with little nger

Five ngers grab

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Appendix C Questionnaire - Part B

Hi!

Thanks for join our study, again! This questionnaire is the last part of this study, we still need

you to ll the following simple questions. Quesions marked by * is required

1. Your ID*

Please write what your input here:

2. Age?

Please write what your input here:

3. Gender?*

Please choose only one of the following:

• Male

• Female

4. Frequency of wearing watches?*

Please use ”Never”, ”Rarely”, ”Sometimes”, ”Often”, ”Always” to answer these questions:

• Frequency of wear it on left hand?

• Frequency of wear it on right hand?

• Frequency without wear a watch?

• Frequency of forget wear it?

5. Do you have use experiences of smart watches?*

Please choose only one of the following:

• Yes

• No

6. Which Model?*

If you choose Yes in question 5, please write this question’s answer:

7. How many watches you own?*

Including traditional watch and smart watch, count them even they have more than one band,

write your answer here:

8. Do you still using watch in your daily life?*

Please choose only one of the following:

• Yes

• No

9. For what reason?*

Please write what your input here:

10. Do you considering to buy a smart watch?*

Please choose only one of the following:

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

• Yes

• No

11. For what reason?*

Please write what your input here:

12. What is the most signicant option of using smart watch for you?*

Please choose only one of the following:

• Price

• Battery

• Habit

• Others, please write it down

13. If you have any comments for this study, we glad to here your voice. Please write

your comments here

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Bibliography

[1] Jiang H, Chen X, Zhang S, et al. Software for Wearable Devices: Challenges and Opportuni-

ties [J/OL]. CoRR. 2015, abs/1504.00747. http://arxiv.org/abs/1504.00747.

[2] Dong S. Progress and Challenge of Human-Computer Interaction [J]. Journal of Computer-Aided

Design and Computer Graphics. 2004, 16 (1): 1–13.

[3] Yue W, Dong S, Wang Y, et al. Study on Human-Computer Interaction Framework of Pervasive

Computing [J]. Chinese Journal of Computers. 2004, 27 (12): 1657–1664.

[4] Hudson S E, Manko J. Concepts, Values, and Methods for Technical Human–Computer Inter-

action Research [M] // Judith S Olson W A K. Ways of Knowing in HCI. Springer, 2014: 2014:

69–93.

[5] Yan Liu. Smart Watch Interaction Design [J]. Science and Innovation. 2015 (8): 73–73.

[6] Chen S. Adapt User Interface Research based on Context-Aware Mobile Devices [D]. [S. l.]: Zhe-

jiang University, 2009.

[7] Fu A. Adapt User Interface Research based on Wearable Computing Device [D]. [S. l.]: University

of Electronic Science and Technology of China, 2006.

[8] Apple Inc. Apple Watch Human Interface Guidelines [EB/OL]. [2016-03-17]. https://developer.

apple.com/watch/human-interface-guidelines/.

[9] Wobbrock J O, Wilson A D, Li Y. Gestures without libraries, toolkits or training: a $1 recognizer

for user interface prototypes [C]. In Proceedings of the 20th annual ACM symposium on User

interface software and technology. 2007: 159–168.

[10] Anthony L, Wobbrock J O. A lightweight multistroke recognizer for user interface prototypes [C].

In Proceedings of Graphics Interface 2010. 2010: 245–252.

[11] Li Q, Fang Z, Sheng M, et al. Gesture Technique and Its Application in Human-Computer Inter-

action [J]. Chinese Journal of Ergonomics. 2002, 8 (1): 27–29.

[12] Chen Y, Ou C, Guo Z. Space interactions based on monocular vision and simple gestures [J].

Journal of Southwest University for Nationalities(Natrual Science Edition). 2014, 40 (6): 871–876.

[13] Di H. 3D Gesture Tracking and Its Application in Human-Computer Interaction [D]. [S. l.]: Nanjing

University, 2011.

[14] Hou W, Wu C. Reasearch of Smart Watch Gesture Interaction Based on Data Analysis [J]. Pack-

aging Engineering. 2015, 36 (22): 13–16.

[15] Vatavu R-D, Zaiti I-A. Leap Gestures for TV: Insights from an Elicitation Study [C/OL]. In

Proceedings of the 2014 ACM International Conference on Interactive Experiences for TV and

Online Video. New York, NY, USA, 2014: 131–138. http://doi.acm.org/10.1145/2602299.

2602316.

[16] Loclair C, Gustafson S, Baudisch P. PinchWatch: a wearable device for one-handed microinterac-

tions [C]. In MobileHCI Workshop on Ensembles of On-Body Devices. 2010.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

[17] Lv Z, Feng S, Feng L, et al. Extending touch-less interaction on vision based wearable device [C].

In Virtual Reality (VR), 2015 iEEE. 2015: 231–232.

[18] Kerber F, Schardt P, Löchtefeld M. WristRotate: A Personalized Motion Gesture Delimiter for

Wrist-worn Devices [C/OL]. In Proceedings of the 14th International Conference on Mobile and

Ubiquitous Multimedia. New York, NY, USA, 2015: 218–222. http://doi.acm.org/10.1145/

2836041.2836063.

[19] Knibbe J, Martinez Plasencia D, Bainbridge C, et al. Extending Interaction for Smart Watches:

Enabling Bimanual Around Device Control [C/OL]. In CHI ’14 Extended Abstracts on Human

Factors in Computing Systems. New York, NY, USA, 2014: 1891–1896. http://doi.acm.org/

10.1145/2559206.2581315.

[20] Kratz S, Rohs M. Hoverow: Exploring Around-device Interaction with IR Distance Sen-

sors [C/OL]. In Proceedings of the 11th International Conference on Human-Computer Interaction

with Mobile Devices and Services. New York, NY, USA, 2009: 42:1–42:4. http://doi.acm.org/

10.1145/1613858.1613912.

[21] Perrault S T, Lecolinet E, Eagan J, et al. Watchit: Simple Gestures and Eyes-free Interaction for

Wristwatches and Bracelets [C/OL]. In Proceedings of the SIGCHI Conference on Human Factors

in Computing Systems. New York, NY, USA, 2013: 1451–1460. http://doi.acm.org/10.1145/

2470654.2466192.

[22] Ogata M, Imai M. SkinWatch: Skin Gesture Interaction for Smart Watch [C/OL]. In Proceedings

of the 6th Augmented Human International Conference. New York, NY, USA, 2015: 21–24. http:

//doi.acm.org/10.1145/2735711.2735830.

[23] Kim J, He J, Lyons K, et al. The gesture watch: A wireless contact-free gesture based wrist

interface [C]. In Wearable Computers, 2007 11th IEEE International Symposium on. 2007: 15–22.

[24] Chen X A, Grossman T, Wigdor D J, et al. Duet: Exploring Joint Interactions on a Smart

Phone and a Smart Watch [C/OL]. In Proceedings of the SIGCHI Conference on Human Factors

in Computing Systems. New York, NY, USA, 2014: 159–168. http://doi.acm.org/10.1145/

2556288.2556955.

[25] Yang F, Li S, Huang R, et al. MagicWatch: Interacting & Segueing [C/OL]. In Proceedings of the

2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Pub-

lication. New York, NY, USA, 2014: 315–318. http://doi.acm.org/10.1145/2638728.2638848.

[26] Bi X, Li Y, Zhai S. FFitts Law: Modeling Finger Touch with Fitts’ Law [C/OL]. In Proceedings

of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY, USA, 2013:

1363–1372. http://doi.acm.org/10.1145/2470654.2466180.

[27] Boring S, Ledo D, Chen X A, et al. The fat thumb: using the thumb’s contact size for single-handed

mobile interaction [M]. New York, New York, USA: ACM, 2012.

[28] Buschek D, Alt F. TouchML: A Machine Learning Toolkit for Modelling Spatial Touch Targeting

Behaviour [C]. In IUI ’15: Proceedings of the 20th International Conference on Intelligent User

Interfaces. March 2015.

[29] Vogel D, Baudisch P. Shift: A Technique for Operating Pen-based Interfaces Using Touch [C/OL].

In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York,

NY, USA, 2007: 657–666. http://doi.acm.org/10.1145/1240624.1240727.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

[30] Huxpro. Forcify [EB/OL]. [2016-03-16]. https://github.com/huxpro/forcify.

[31] Changkun Ou. AugmentedTouch: Potential Device Behaviour [EB/OL]. [2016-04-02]. https:

//github.com/changkun/AugmentedTouch.

[32] Leap Motion Inc. Leap SDK Documentation [EB/OL]. [2016-03-17]. https://developer.

leapmotion.com/documentation/index.html.

[33] Weichert F, Bachmann D, Rudak B, et al. Analysis of the Accuracy and Robustness of the Leap

Motion Controller [J]. Sensors. 2013, 13: 6380–6393.

[34] Du G, Zhang P, Liu X. Markerless Human-Manipulator Interface Using Leap Motion with Interval

Kalman Filter and Improved Particle Filter [J]. IEEE Transactions on Industrial Informatics. 2016,

PP (99): 1–1.

[35] Garber L. Gestural technology: Moving interfaces in a new direction [J]. Computer. 2013, 46 (10):

22–25.

[36] Xu Z, Zhou M, Shen J, et al. A Interaction technique based on LeapMotion [J]. Journal of Elec-

tronics and Information Technology. 2015, 37 (2): 353–359.

[37] Pan J, Xu K. 3D Gesture Operation Based on LeapMotion [J]. China Science Paper. 2015 (2):

207–212.

[38] Hu H, Yao J, Yang J, et al. Key Point Model for Gesture Estimation Based on Leap Motion [J].

Journal of Computer-Aided Design and Computer Graphics. 2015, 27 (7): 1211–1216.

[39] Marin G, Dominio F, Zanuttigh P. Hand gesture recognition with leap motion and kinect de-

vices [C]. In Image Processing (ICIP), 2014 IEEE International Conference on. 2014: 1565–1569.

[40] Zaiţi I-A, Pentiuc Ş-G, Vatavu R-D. On free-hand TV control: experimental results on user-elicited

gestures with Leap Motion [J]. Personal and Ubiquitous Computing. 2015, 19 (5-6): 821–838.

[41] Apple Inc. App Programming Guide for watchOS [EB/OL]. [2016-04-02]. https:

//developer.apple.com/library/watchos/documentation/General/Conceptual/

WatchKitProgrammingGuide/.

[42] Apple Inc. Watch Connectivity Framework Reference [EB/OL]. [2016-04-02]. https:

//developer.apple.com/library/watchos/documentation/WatchConnectivity/Reference/

WatchConnectivity_framework/.

[43] Apple Inc. The Swift Programming Language(Swift 2.2 Prerelease) [M/OL]. Apple Inc., 2016.

https://itunes.apple.com/cn/book/swift-programming-language/id1002622538.

[44] Apple Inc. Using Swift with Cocoa and Objective-C(Swift 2.2 Prerelease) [M/OL]. Apple Inc.,

2016. https://itunes.apple.com/cn/book/using-swift-cocoa-objective/id1002624212.

[45] Albert W, Tullis T. Measuring the user experience: collecting, analyzing, and presenting usability

metrics [M]. Newnes, 2013.

[46] Faulkner L. Beyond the ve-user assumption: Benets of increased sample sizes in usabil-

ity testing [J/OL]. Behavior Research Methods, Instruments, & Computers: 379–383. http:

//dx.doi.org/10.3758/BF03195514.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

[47] Hwang W, Salvendy G. Number of People Required for Usability Evaluation: The 10±2

Rule [J/OL]. Commun. ACM. 2010, 53 (5): 130–133. http://doi.acm.org/10.1145/1735223.

1735255.

[48] Maceeld R. How to Specify the Participant Group Size for Usability Studies: A Practitioner’s

Guide [J/OL]. J. Usability Studies. 2009, 5 (1): 34–45. http://dl.acm.org/citation.cfm?id=

2835425.2835429.

[49] Medlock M C, Wixon D, Terrano M, et al. Using the RITE method to improve products: A

denition and a case study [J]. Usability Professionals Association. 2002, 51.

[50] Pernice K, Nielsen J. Eyetracking methodology: How to conduct and evaluate usability studies

using eyetracking [J]. Nielsen Norman Group Technical Report. 2009.

[51] Bailly G, Müller J, Rohs M, et al. ShoeSense: A New Perspective on Gestural Interaction and

Wearable Applications [C/OL]. In Proceedings of the SIGCHI Conference on Human Factors in

Computing Systems. New York, NY, USA, 2012: 1239–1248. http://doi.acm.org/10.1145/

2207676.2208576.

[52] Card S K, Moran T P, Newell A. The Keystroke-level Model for User Performance Time with

Interactive Systems [J/OL]. Commun. ACM. 1980, 23 (7): 396–410. http://doi.acm.org/10.

1145/358886.358895.

[53] Thalmic Labs Inc. Myo Ocial Site [EB/OL]. [2016-04-05]. https://www.myo.com.

[54] Ou C, Guo Z, Shi M, et al. Research and Implementation of Interaction Perception Center Based

on Machine Learning and Cloud Computing [R/OL]. 2015. https://changkun.de/blog/ipc/.

[55] Yang Y, Chae S, Shim J, et al. EMG Sensor-based Two-Hand Smart Watch Interaction [C/OL].

In Adjunct Proceedings of the 28th Annual ACM Symposium on User Interface Software & Tech-

nology. New York, NY, USA, 2015: 73–74. http://doi.acm.org/10.1145/2815585.2815724.

[56] Fu Q. Motion Interaction Platform Implementation [D]. [S. l.]: University of Electronic Science

and Technology of China, 2015.

[57] Zhu Z. Research on Smart Watch Conext [J]. China New Technologies and New Products. 2015 (7):

10–12.

西南民族大学本科毕业生论文欧长坤：智能手表的非接触式备择交互模式的设计

Acknowledgments

This thesis has nished under Prof.Yaxi’s guidance. She was the most important teacher in my life

and she gave me great guidance for everything during my four years of undergraduate learning. With

her help, I took part in the American Mathematical Contest in Modeling when I was a freshman, and

also applied an entrepreneurship students projects in the second year. Then she supervised me how to

write a research papers, gave me a chance to study at University of Munich for exchange semester, and

now reviewed my graduate thesis.

Prof.Yaxi and her rigorous, realistic and serious encourage me to keep continue broad my horizon

of knowledge, ability of research, even inuenced the roads of my future life. These are any other

teachers who helped me will never reached. So, rst of all, I would like to send the greatest respect

and sincere gratitude to my adviser Prof. Yaxi.

Then I would like to thank all my family, especially my parents. I will never archive this without

their continuous support and inclusion throughout my entire education, they are my shield of every-

thing. I appreciate my friends, we motivate each other, and pursue a common ideal together, and I

appreciate the teachers who helped, cared, guided for me, I have been through an enriching university

life because of them.

Lastly, I still need to appreciate my advisor Prof. Andreas Butz, Prof. Heinrich Hußmann and PhD

student Daniel Buschek who works at University of Munich. They helped me experienced an enjoyable

exchange semester for study abroad. Besides, There is a special thanks for Prof. Butz because he

agreed to take over the co-advisor of my thesis and reviewed the english version of my thesis.

With the end of this article, my undergraduate studies will be completed, I will towards to next

stage of my entire life with my family, teachers’ encouragement and their expectation.

Wish I could keep forever youthful, forever weeping, cheers!