Context-detection API for Android developed as a university project
Du kannst nicht mehr als 25 Themen auswählen Themen müssen mit entweder einem Buchstaben oder einer Ziffer beginnen. Sie können Bindestriche („-“) enthalten und bis zu 35 Zeichen lang sein.

final.lyx 59KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077107810791080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104110511061107110811091110111111121113111411151116111711181119112011211122112311241125112611271128112911301131113211331134113511361137113811391140114111421143114411451146114711481149115011511152115311541155115611571158115911601161116211631164116511661167116811691170117111721173117411751176117711781179118011811182118311841185118611871188118911901191119211931194119511961197119811991200120112021203120412051206120712081209121012111212121312141215121612171218121912201221122212231224122512261227122812291230123112321233123412351236123712381239124012411242124312441245124612471248124912501251125212531254125512561257125812591260126112621263126412651266126712681269127012711272127312741275127612771278127912801281128212831284128512861287128812891290129112921293129412951296129712981299130013011302130313041305130613071308130913101311131213131314131513161317131813191320132113221323132413251326132713281329133013311332133313341335133613371338133913401341134213431344134513461347134813491350135113521353135413551356135713581359136013611362136313641365136613671368136913701371137213731374137513761377137813791380138113821383138413851386138713881389139013911392139313941395139613971398139914001401140214031404140514061407140814091410141114121413141414151416141714181419142014211422142314241425142614271428142914301431143214331434143514361437143814391440144114421443144414451446144714481449145014511452145314541455145614571458145914601461146214631464146514661467146814691470147114721473147414751476147714781479148014811482148314841485148614871488148914901491149214931494149514961497149814991500150115021503150415051506150715081509151015111512151315141515151615171518151915201521152215231524152515261527152815291530153115321533153415351536153715381539154015411542154315441545154615471548154915501551155215531554155515561557155815591560156115621563156415651566156715681569157015711572157315741575157615771578157915801581158215831584158515861587158815891590159115921593159415951596159715981599160016011602160316041605160616071608160916101611161216131614161516161617161816191620162116221623162416251626162716281629163016311632163316341635163616371638163916401641164216431644164516461647164816491650165116521653165416551656165716581659166016611662166316641665166616671668166916701671167216731674167516761677167816791680168116821683168416851686168716881689169016911692169316941695169616971698169917001701170217031704170517061707170817091710171117121713171417151716171717181719172017211722172317241725172617271728172917301731173217331734173517361737173817391740174117421743174417451746174717481749175017511752175317541755175617571758175917601761176217631764176517661767176817691770177117721773177417751776177717781779178017811782178317841785178617871788178917901791179217931794179517961797179817991800180118021803180418051806180718081809181018111812181318141815181618171818181918201821182218231824182518261827182818291830183118321833183418351836183718381839184018411842184318441845184618471848184918501851185218531854185518561857185818591860186118621863186418651866186718681869187018711872187318741875187618771878187918801881188218831884188518861887188818891890189118921893189418951896189718981899190019011902190319041905190619071908190919101911191219131914191519161917191819191920192119221923192419251926192719281929193019311932193319341935193619371938193919401941194219431944194519461947194819491950195119521953195419551956195719581959196019611962196319641965196619671968196919701971197219731974197519761977197819791980198119821983198419851986198719881989199019911992199319941995199619971998199920002001200220032004200520062007200820092010201120122013201420152016201720182019202020212022202320242025202620272028202920302031203220332034203520362037203820392040204120422043204420452046204720482049205020512052205320542055205620572058205920602061206220632064206520662067206820692070207120722073207420752076207720782079208020812082208320842085208620872088208920902091209220932094209520962097209820992100210121022103210421052106210721082109211021112112211321142115211621172118211921202121212221232124212521262127212821292130213121322133213421352136213721382139214021412142214321442145214621472148214921502151215221532154215521562157215821592160216121622163216421652166216721682169217021712172217321742175217621772178217921802181218221832184218521862187218821892190219121922193219421952196219721982199220022012202220322042205220622072208220922102211221222132214221522162217221822192220222122222223222422252226222722282229
  1. #LyX 1.6.5 created this file. For more info see http://www.lyx.org/
  2. \lyxformat 345
  3. \begin_document
  4. \begin_header
  5. \textclass article
  6. \use_default_options true
  7. \language english
  8. \inputencoding auto
  9. \font_roman default
  10. \font_sans default
  11. \font_typewriter default
  12. \font_default_family default
  13. \font_sc false
  14. \font_osf false
  15. \font_sf_scale 100
  16. \font_tt_scale 100
  17. \graphics default
  18. \paperfontsize default
  19. \spacing single
  20. \use_hyperref false
  21. \papersize a4paper
  22. \use_geometry false
  23. \use_amsmath 1
  24. \use_esint 1
  25. \cite_engine basic
  26. \use_bibtopic false
  27. \paperorientation portrait
  28. \secnumdepth 3
  29. \tocdepth 3
  30. \paragraph_separation skip
  31. \defskip medskip
  32. \quotes_language english
  33. \papercolumns 1
  34. \papersides 1
  35. \paperpagestyle default
  36. \tracking_changes false
  37. \output_changes false
  38. \author ""
  39. \author ""
  40. \end_header
  41. \begin_body
  42. \begin_layout Title
  43. Context-Aware API for Android Devices
  44. \end_layout
  45. \begin_layout Author
  46. Chris Smith
  47. \begin_inset Newline newline
  48. \end_inset
  49. Supervisor: Naranker Dulay
  50. \end_layout
  51. \begin_layout Date
  52. Summer 2010
  53. \end_layout
  54. \begin_layout Standard
  55. All materials can be accessed electronically at:
  56. \end_layout
  57. \begin_layout Standard
  58. /homes/cs106/....
  59. \end_layout
  60. \begin_layout Standard
  61. \begin_inset Newpage pagebreak
  62. \end_inset
  63. \end_layout
  64. \begin_layout Abstract
  65. In recent years location-based services have seen a dramatic increase in
  66. adoption, and all modern smartphone platforms have integrated services
  67. to facilitate the creation location-aware applications.
  68. Such applications enhance the experience of users, using location to modify
  69. content or alter the behaviour of the application to better suit the circumstan
  70. ces.
  71. \end_layout
  72. \begin_layout Abstract
  73. The product of this project is a
  74. \emph on
  75. context
  76. \emph default
  77. -aware API for the Android platform.
  78. This allows applications to augment the already available location data
  79. with extra
  80. \emph on
  81. context
  82. \emph default
  83. about the user's situation - primarily their current activity.
  84. It also develops an algorithm for recognising
  85. \emph on
  86. places
  87. \emph default
  88. which are relevant to the user, and monitoring which activities are performed
  89. in
  90. \emph on
  91. journeys
  92. \emph default
  93. between those places, thus enabling predictions of the user's likely destinatio
  94. n based on their activity.
  95. \end_layout
  96. \begin_layout Abstract
  97. Research into other methods of annotating context was conducted, and it
  98. was found that most potential sources of context information either produced
  99. little or no information, or were too battery-draining to perform in a
  100. real world environment with current technology.
  101. Much effort was placed into optimising the API to have as small effect
  102. on battery life as possible.
  103. \end_layout
  104. \begin_layout Abstract
  105. A context-aware API was successfully produced, along with a collection of
  106. applications which use the API in order to demonstrate its features or
  107. provide example use cases.
  108. \end_layout
  109. \begin_layout Standard
  110. \begin_inset Newpage pagebreak
  111. \end_inset
  112. \end_layout
  113. \begin_layout LyX-Code
  114. \begin_inset CommandInset toc
  115. LatexCommand tableofcontents
  116. \end_inset
  117. \end_layout
  118. \begin_layout LyX-Code
  119. \begin_inset Newpage pagebreak
  120. \end_inset
  121. \end_layout
  122. \begin_layout Part
  123. Introduction
  124. \end_layout
  125. \begin_layout Standard
  126. \begin_inset Newpage pagebreak
  127. \end_inset
  128. \end_layout
  129. \begin_layout Part
  130. Background
  131. \end_layout
  132. \begin_layout Standard
  133. Context- or Activity-Aware devices is an area currently under lots of research.
  134. There are many and varied applications of activity-aware devices, ranging
  135. from personal fitness and healthcare to training factory workers or merely
  136. playing music.
  137. \end_layout
  138. \begin_layout Standard
  139. While this research is going on, there has been a huge expansion in the
  140. ownership, use and power of mobile telephones.
  141. Mobile telephones are so ubiquitous and now come with such a large sensor
  142. platform that they are the obvious choice for implementing activity-aware
  143. technologies for use in day-to-day life.
  144. \end_layout
  145. \begin_layout Standard
  146. This project aims to make a context-aware API available on an open mobile
  147. platform, which will enable developers to start adding context-aware functional
  148. ity to their applications without the extremely large overhead of writing
  149. a logger and classifier themselves, or re-engineering the application to
  150. use an existing context-aware framework if one is available.
  151. \end_layout
  152. \begin_layout Section
  153. Applications
  154. \end_layout
  155. \begin_layout Standard
  156. There are many documented applications of activity-aware systems, and current
  157. research efforts which bring the technology to mobile telephones will only
  158. serve to lengthen this list.
  159. \end_layout
  160. \begin_layout Standard
  161. The canonical example for activity-awareness, especially on mobile telephones,
  162. is modeling the user's
  163. \begin_inset Quotes eld
  164. \end_inset
  165. interruptibility
  166. \begin_inset Quotes erd
  167. \end_inset
  168. \begin_inset CommandInset citation
  169. LatexCommand cite
  170. key "Siewiorek2003,Raento2005"
  171. \end_inset
  172. .
  173. This allows the software to know whether it's appropriate (or "polite")
  174. to disturb the user, and can advise the user's contacts when they are busy.
  175. It can also be used to create a
  176. \begin_inset Quotes eld
  177. \end_inset
  178. smart answering machine
  179. \begin_inset Quotes erd
  180. \end_inset
  181. \begin_inset CommandInset citation
  182. LatexCommand cite
  183. key "Hudson2003"
  184. \end_inset
  185. which can selectively direct calls straight to an answering machine if
  186. the user is engaged in an "uninterpretable" activity and the call does
  187. not appear to be important.
  188. These allow the user's mobile telephone to better approximate human behaviour
  189. - when approaching someone in person it is normally quite easy to determine
  190. whether it would be polite or necessary to disturb them, based on their
  191. demeanour, activity, and the urgency of your request; when picking up the
  192. telephone it is not possible at all without assistance from an activity-aware
  193. system.
  194. \end_layout
  195. \begin_layout Standard
  196. The current implementations of these ideas have several problems, however.
  197. The more interesting research
  198. \begin_inset CommandInset citation
  199. LatexCommand cite
  200. key "Hudson2003"
  201. \end_inset
  202. requires a static camera fixed in an office to observe user behaviour,
  203. instead of implementing it directly on a telephone, which obviously constrains
  204. its usefulness.
  205. Of the two solutions actually targeted at mobile telephones, one
  206. \begin_inset CommandInset citation
  207. LatexCommand cite
  208. key "Siewiorek2003"
  209. \end_inset
  210. requires bulky custom hardware which the user must carry on their belt,
  211. and the other
  212. \begin_inset CommandInset citation
  213. LatexCommand cite
  214. key "Raento2005"
  215. \end_inset
  216. does not expose an API to other applications and only surfaces the context-aware
  217. functionality in two small applications whose focus is on social interaction
  218. rather than improving the user's experience of the telephone locally.
  219. This project will aim to bring the ideas of these to generic hardware (an
  220. Android mobile telephone), and to provide an API which other applications
  221. can harness.
  222. \end_layout
  223. \begin_layout Standard
  224. One use particularly suitable for mobile phones is dynamic adaptation of
  225. the device's settings based on the user's current activity and context
  226. \begin_inset CommandInset citation
  227. LatexCommand cite
  228. key "Schmidt1999"
  229. \end_inset
  230. .
  231. When a user is walking the device can dynamically increase the font size
  232. to make it easier to read with an unsteady hand, and correspondingly decrease
  233. it when the user is stationary.
  234. In a similar fashion, the brightness of the backlight can be altered based
  235. on the ambient light level, and the ringer volume altered according to
  236. the noise level.
  237. Unfortunately this research did not progress beyond a feasibility study
  238. and was implemented on a Nokia 6110, which is severely outdated by today's
  239. standards.
  240. \end_layout
  241. \begin_layout Standard
  242. Another popular area for activity-aware systems is in healthcare.
  243. Such systems can be used to monitor vulnerable people as they go about
  244. day to day activities to ensure that they're not in trouble - several systems
  245. \begin_inset CommandInset citation
  246. LatexCommand cite
  247. key "Song2005,Maurer2006"
  248. \end_inset
  249. can be used to monitor elderly persons and summon help if it is detected
  250. that they have fallen.
  251. Another healthcare application
  252. \begin_inset CommandInset citation
  253. LatexCommand cite
  254. key "Tentori2008"
  255. \end_inset
  256. allows nurses to remotely monitor the activities of their patients in a
  257. hospital ward, allowing them to respond to problems and keep up-to-date
  258. with their patients' well-being while not physically present.
  259. Activity-aware applications have also been used to try to encourage users
  260. to be more healthy; one novel application records the day-to-day fitness
  261. activities a user performs and uses this as a basis for a virtual
  262. \begin_inset Quotes eld
  263. \end_inset
  264. garden
  265. \begin_inset Quotes erd
  266. \end_inset
  267. that blossoms or wilts according to how much the user works out in a week
  268. \begin_inset CommandInset citation
  269. LatexCommand cite
  270. key "Consolvo2008"
  271. \end_inset
  272. .
  273. \end_layout
  274. \begin_layout Standard
  275. As well as monitoring activities which the user is familiar with, activity-aware
  276. systems can also be used to assist users in learning new activities.
  277. One application
  278. \begin_inset CommandInset citation
  279. LatexCommand cite
  280. key "Stiefmeier2008"
  281. \end_inset
  282. monitors the activities of trainee workers in a car manufacturing plant,
  283. and helps to provide a link between theoretical classroom-based training
  284. and practical work.
  285. The activity-aware system can offer advice to the workers that's directly
  286. related to the current task they're performing, and can even monitor their
  287. activities for compliance with procedures and give them a score afterwards.
  288. \end_layout
  289. \begin_layout Standard
  290. While the research into healthcare and training applications present novel
  291. uses of activity-aware systems, the applications themselves are not really
  292. applicable to a mobile device or the scope of this project.
  293. The research does, however, describe the techniques used in those applications
  294. for activity classification and should prove useful in that respect.
  295. \end_layout
  296. \begin_layout Standard
  297. Other areas of research include making activity-aware suggestions to the
  298. user
  299. \begin_inset CommandInset citation
  300. LatexCommand cite
  301. key "Bellotti2008"
  302. \end_inset
  303. , or issuing reminders or alerts based on the user's activity
  304. \begin_inset CommandInset citation
  305. LatexCommand cite
  306. key "Schilit1994"
  307. \end_inset
  308. .
  309. One example of the latter is an activity-aware system that detects when
  310. the user is making coffee, and plays a sound on a remote computer to alert
  311. thirsty coworkers to the fact.
  312. Sound isn't only limited to alerts, however: the
  313. \noun on
  314. XPOD
  315. \noun default
  316. \begin_inset CommandInset citation
  317. LatexCommand cite
  318. key "Dornbush2005"
  319. \end_inset
  320. project is an activity-aware music player, which tailors the music being
  321. played to the user's current activity based on their past ratings.
  322. This type of activity-aware device presents a much greater level of personalisa
  323. tion than previously possible, and making this type of customisation available
  324. to mobile telephone users and application developers will surely result
  325. in many new applications.
  326. \end_layout
  327. \begin_layout Section
  328. Inferring activity
  329. \end_layout
  330. \begin_layout Standard
  331. There are three general phases in most context-aware systems: a sensing
  332. component, which reads or receives raw sensor data relating to the user's
  333. environment or activity; a feature extraction component, which analysis
  334. the sensor data and identifies a set of features from that data; and a
  335. classification component, which uses the extracted features to reason about
  336. the user's activity
  337. \begin_inset CommandInset citation
  338. LatexCommand cite
  339. key "Choudhury2008"
  340. \end_inset
  341. .
  342. Each of these components will be expanded on below.
  343. Depending on the method of classification, some initial or continuous training
  344. may be required, and this is also considered below.
  345. \end_layout
  346. \begin_layout Subsection
  347. Sensors and devices
  348. \end_layout
  349. \begin_layout Standard
  350. At the basis of activity-recognition are the physical hardware sensors.
  351. The most commonly used sensor is the accelerometer, which outputs the accelerat
  352. ion of the sensor
  353. \begin_inset Foot
  354. status collapsed
  355. \begin_layout Plain Layout
  356. and thus the device to which it's attached, and therefore the person using
  357. the device
  358. \end_layout
  359. \end_inset
  360. along a certain axis.
  361. There is extensive research on using accelerometers to classify activities
  362. such as walking
  363. \begin_inset CommandInset citation
  364. LatexCommand cite
  365. key "Mathie2004,Garakani2009"
  366. \end_inset
  367. (including whether or not the subject is walking on flat ground or up and
  368. down stairs
  369. \begin_inset CommandInset citation
  370. LatexCommand cite
  371. key "Caros2005"
  372. \end_inset
  373. ), running
  374. \begin_inset CommandInset citation
  375. LatexCommand cite
  376. key "Garakani2009"
  377. \end_inset
  378. , falling
  379. \begin_inset CommandInset citation
  380. LatexCommand cite
  381. key "Mathie2004"
  382. \end_inset
  383. , sitting
  384. \begin_inset CommandInset citation
  385. LatexCommand cite
  386. key "Mathie2004,Garakani2009"
  387. \end_inset
  388. , cycling
  389. \begin_inset CommandInset citation
  390. LatexCommand cite
  391. key "Garakani2009"
  392. \end_inset
  393. , etc.
  394. \end_layout
  395. \begin_layout Standard
  396. Accelerometers can be combined with magnetic field sensors to convert the
  397. relative acceleration of the device into an absolute acceleration with
  398. respect to the Earth.
  399. This is particularly important when the sensor platform - a mobile telephone
  400. - can be oriented in different ways while performing the same activity.
  401. There are, for example, four different ways to comfortably fit a typical
  402. smartphone into a trouser pocket, made up of two possible orientations
  403. around two axis
  404. \begin_inset Foot
  405. status collapsed
  406. \begin_layout Plain Layout
  407. screen facing towards or away from the body, and the device oriented the
  408. right way up or upside down
  409. \end_layout
  410. \end_inset
  411. .
  412. With just raw accelerometer data these four orientations would lead to
  413. different data for the same activity, but with the combination of magnetic
  414. field sensors we can normalise them.
  415. \end_layout
  416. \begin_layout Standard
  417. Smartphones also come equipped with a microphone and GSM stack (prerequisites
  418. for a telephone conversation!), and commonly a camera, geolocation API
  419. (usually backed by GPS) and Bluetooth stack.
  420. With the exception of the latter two, these types of sensors are not particular
  421. ly well explored for their use in context-aware systems at present.
  422. It is easy to reason how each would be useful - a microphone can reveal
  423. the ambient noise, which could indicate the difference between sitting
  424. in a library and a bar; the camera likewise can reveal the lighting conditions
  425. (if the device is not in a pocket or bag).
  426. The GSM stack can provide rough location information and also a signal
  427. strength to one or more cell towers; the signal strength will vary both
  428. with the user's proximity to the cell tower and the environment around
  429. them - being inside will degrade the signal more than being in open air,
  430. for example - so may provide vital clues to a context-aware system.
  431. One aspect of this project will be to research how the microphone, camera
  432. and GSM stack can be used to enhance existing activity classification algorithm
  433. s.
  434. \end_layout
  435. \begin_layout Standard
  436. Current research on location information and Bluetooth device proximity
  437. is summarised in section
  438. \begin_inset CommandInset ref
  439. LatexCommand ref
  440. reference "sec:Location-analysis"
  441. \end_inset
  442. (p
  443. \begin_inset CommandInset ref
  444. LatexCommand pageref
  445. reference "sec:Location-analysis"
  446. \end_inset
  447. ) and
  448. \begin_inset CommandInset ref
  449. LatexCommand ref
  450. reference "sec:Bluetooth"
  451. \end_inset
  452. (p
  453. \begin_inset CommandInset ref
  454. LatexCommand pageref
  455. reference "sec:Bluetooth"
  456. \end_inset
  457. ) respectively.
  458. \end_layout
  459. \begin_layout Subsection
  460. Feature detection
  461. \end_layout
  462. \begin_layout Standard
  463. It is not possible to reason directly about raw sensor inputs, so the next
  464. step in inferring activities is to extract useful
  465. \emph on
  466. features
  467. \emph default
  468. from the raw input.
  469. Features are usually mathematical properties of the input data, such as
  470. the difference between the minimum and maximum data point in a given time
  471. frame.
  472. Most classifiers use an extremely large number of features -
  473. \begin_inset CommandInset citation
  474. LatexCommand citet
  475. key "Hein2008"
  476. \end_inset
  477. detect 562 different features from their inputs.
  478. \end_layout
  479. \begin_layout Standard
  480. Some of the more commonly used features in activity-recognition systems
  481. are: mean, standard deviation, energy, entropy, correlation between axis,
  482. and discrete FFT coefficients
  483. \begin_inset CommandInset citation
  484. LatexCommand cite
  485. key "Huynh2005"
  486. \end_inset
  487. .
  488. Obviously, not all features are of equal value.
  489. FFT coefficients are generally very good indicators of activity, but the
  490. ideal coefficients and window sizes vary depending on the exact activity
  491. that is being detected.
  492. Likewise, the choice of other features to give the best recognition rate
  493. varies depending on the activity being detected
  494. \begin_inset CommandInset citation
  495. LatexCommand cite
  496. key "Huynh2005"
  497. \end_inset
  498. .
  499. \end_layout
  500. \begin_layout Standard
  501. As the sensor data is received continuously, it needs to be partitioned
  502. somehow before features are extracted.
  503. Most implementations use a sliding window approach with a 50% overlap between
  504. windows
  505. \begin_inset CommandInset citation
  506. LatexCommand cite
  507. key "Bao2004"
  508. \end_inset
  509. .
  510. A window size of 10 seconds with a 50% overlap would result in one set
  511. of features being computed every 5 seconds.
  512. The window size is normally selected to correspond to a pre-defined number
  513. of samples to enable fast computation of certain features - most notably
  514. FFTs
  515. \begin_inset CommandInset citation
  516. LatexCommand cite
  517. key "Bao2004"
  518. \end_inset
  519. .
  520. \end_layout
  521. \begin_layout Standard
  522. One challenge will be determining a set of features that are robust enough
  523. to perform activity analysis on, but are sufficiently inexpensive to calculate
  524. continually on a mobile device, where CPU speed is limited and excessive
  525. usage results in undesirable higher battery consumption.
  526. \end_layout
  527. \begin_layout Subsection
  528. Training
  529. \end_layout
  530. \begin_layout Standard
  531. In order to meaningfully classify and label activities, some kind of training
  532. generally needs to be performed beforehand.
  533. The choice of classifier affects how much offline analysis has to be done
  534. on the training set, and whether or not it can be adapted at run-time.
  535. \end_layout
  536. \begin_layout Standard
  537. One might expect that training would best be performed in a controlled environme
  538. nt, to reduce external influences on the user, but subjects in a laboratory
  539. setting are much more self-conscious about their movements, and this manifests
  540. itself in the data collected.
  541. Walking in a laboratory tends to produce acceleration data showing a consistent
  542. gait cycle which can be split into distinct phases, whereas walking in
  543. an uncontrolled setting produces data showing large fluctuations in gait
  544. phases and length.
  545. This means that classifiers trained on laboratory data may achieve a much
  546. lower accuracy when deployed in natural conditions
  547. \begin_inset CommandInset citation
  548. LatexCommand cite
  549. key "Bao2004"
  550. \end_inset
  551. .
  552. \end_layout
  553. \begin_layout Subsection
  554. Classification
  555. \end_layout
  556. \begin_layout Standard
  557. The classification step involves feeding the features for frame into some
  558. kind of machine learning algorithm which can, using training data
  559. \begin_inset Foot
  560. status collapsed
  561. \begin_layout Plain Layout
  562. and any offline analysis made of that data
  563. \end_layout
  564. \end_inset
  565. , determine which activity the feature-set most like represents.
  566. There are many different algorithms that can be used to perform the classificat
  567. ion, some of which are discussed below.
  568. \end_layout
  569. \begin_layout Subsubsection
  570. Decision trees
  571. \end_layout
  572. \begin_layout Standard
  573. Decision trees are possibly one of the simplest approaches possible
  574. \begin_inset CommandInset citation
  575. LatexCommand cite
  576. key "Hudson2003"
  577. \end_inset
  578. .
  579. A tree is constructed such that each node contains a test function, with
  580. branches for each possible discrete outcome of the function.
  581. This allows data to be classified with a
  582. \begin_inset Quotes eld
  583. \end_inset
  584. divide and conquer
  585. \begin_inset Quotes erd
  586. \end_inset
  587. approach.
  588. While high accuracy is possible in some circumstances
  589. \begin_inset CommandInset citation
  590. LatexCommand cite
  591. key "Hudson2003"
  592. \end_inset
  593. , there are several drawbacks to decision trees: a plain decision tree has
  594. no way to model uncertainty - in an activity-aware system there will always
  595. be a degree of uncertainty as to the classification, and being able to
  596. measure this is an important tool.
  597. They also have an inductive bias which leads to a preference for the most
  598. general solution, and in most cases this generalisation causes many false
  599. results
  600. \begin_inset CommandInset citation
  601. LatexCommand cite
  602. key "Shen2004"
  603. \end_inset
  604. .
  605. \end_layout
  606. \begin_layout Standard
  607. Decision trees require the structure of the tree and the test functions
  608. for each node to be determined during training.
  609. They do not lend themselves to minor on-the-fly modifications or new activities
  610. that are not part of the training set.
  611. \end_layout
  612. \begin_layout Subsubsection
  613. Neural networks
  614. \end_layout
  615. \begin_layout Standard
  616. Neural networks are based on an extremely simplified model of the brain.
  617. The network consists of layers of neurons, and each neuron performs a simple
  618. arithmetic operation on its inputs.
  619. This normally consists of taking each of its inputs, multiplying it by
  620. a weight, and then summing all of the weighted inputs together; the resulting
  621. figure then becomes the neuron's output, and the input to one or more nodes
  622. in the next layer.
  623. \end_layout
  624. \begin_layout Standard
  625. A network consists of a layer of input neurons, a layer containing one or
  626. more output neurons, and one or more layers of
  627. \begin_inset Quotes eld
  628. \end_inset
  629. hidden
  630. \begin_inset Quotes erd
  631. \end_inset
  632. neurons in between.
  633. The number of
  634. \begin_inset Quotes eld
  635. \end_inset
  636. hidden
  637. \begin_inset Quotes erd
  638. \end_inset
  639. layers, and the number of neurons within those layers must be chosen before
  640. training of the network begins.
  641. The training process will then determine the weights for each link in the
  642. network.
  643. The choice of number of layers poses a problem when designing a network,
  644. as too small a number can cripple the power of the network, but too large
  645. can cause it to be too expensive to evaluate and can possibly lead to it
  646. memorising erroneous data
  647. \begin_inset CommandInset citation
  648. LatexCommand cite
  649. key "Dornbush2005"
  650. \end_inset
  651. .
  652. \end_layout
  653. \begin_layout Standard
  654. Neural networks, however, do provide good accuracy and could potentially
  655. (although not easily) be modified on-the-fly to cope with new activities.
  656. \end_layout
  657. \begin_layout Subsubsection
  658. Genetic algorithms
  659. \end_layout
  660. \begin_layout Standard
  661. Genetic algorithms use the principle of natural selection to 'evolve' a
  662. solution to a problem.
  663. A set of random solutions are created, and a pre-defined fitness function
  664. is used to determine their relative worth.
  665. The best solutions are then combined together to produce the next generation
  666. of solutions, in a manner roughly analogous to reproduction in animals.
  667. Small
  668. \begin_inset Quotes eld
  669. \end_inset
  670. mutations
  671. \begin_inset Quotes erd
  672. \end_inset
  673. are also introduced into each generation to counter the effect of local
  674. maxima being reached.
  675. \end_layout
  676. \begin_layout Standard
  677. Genetic algorithms can be combined with other techniques such as neural
  678. networks - the weights in the neural network can be
  679. \begin_inset Quotes eld
  680. \end_inset
  681. evolved
  682. \begin_inset Quotes erd
  683. \end_inset
  684. using genetic algorithms to create a neural network which is good as satisfying
  685. the fitness function.
  686. \end_layout
  687. \begin_layout Standard
  688. The drawback of genetic algorithms is the need for a fitness function -
  689. the network will only ever be as good as the fitness function, and if you
  690. have a way to define what makes a good network you could in most cases
  691. hardcode the solution instead of evolving a network to satisfy it.
  692. \end_layout
  693. \begin_layout Subsubsection
  694. Instance-based learning
  695. \end_layout
  696. \begin_layout Standard
  697. Instance-based learning (IBL)
  698. \begin_inset CommandInset citation
  699. LatexCommand cite
  700. key "Witten2000"
  701. \end_inset
  702. algorithms are a class of
  703. \begin_inset Quotes eld
  704. \end_inset
  705. lazy
  706. \begin_inset Quotes erd
  707. \end_inset
  708. algorithms.
  709. They perform classification based on previously observed instances that
  710. have already been classified.
  711. There is no training required for IBLs, they're extremely adept at adapting
  712. to new scenarios, and they have a very low error rate
  713. \begin_inset CommandInset citation
  714. LatexCommand cite
  715. key "Dornbush2005"
  716. \end_inset
  717. which makes them ideal for activity-recognition.
  718. \end_layout
  719. \begin_layout Standard
  720. One particular type of IBL algorithm which is frequently seen in activity-aware
  721. research is the K-Nearest Neighbour (KNN) algorithm
  722. \begin_inset CommandInset citation
  723. LatexCommand cite
  724. key "Han2006"
  725. \end_inset
  726. .
  727. With the KNN algorithm, each sample is treated as a vector, and the distance
  728. \begin_inset Foot
  729. status collapsed
  730. \begin_layout Plain Layout
  731. the euclidean distance is usually used, but any metric will suffice
  732. \end_layout
  733. \end_inset
  734. between the sample and the existing instances is calculated.
  735. The sample is then classified according to the classification of the majority
  736. of its
  737. \begin_inset Formula $k$
  738. \end_inset
  739. nearest neighbours.
  740. \end_layout
  741. \begin_layout Standard
  742. One drawback of IBLs is that each new instance tends to be remembered for
  743. future use, which eventually results in large amounts of memory consumption
  744. and complexity when comparing distances of new samples.
  745. This can be partially overcome by only storing instances which would affect
  746. the classification of new samples
  747. \begin_inset CommandInset citation
  748. LatexCommand cite
  749. key "Witten2000"
  750. \end_inset
  751. .
  752. \end_layout
  753. \begin_layout Standard
  754. The KNN algorithm can be easily extended to support dynamic classification
  755. of new types of activities - if a sample is not within a certain distance
  756. of sufficient other samples, it can be classified as a new type of activity.
  757. \end_layout
  758. \begin_layout Subsubsection
  759. Conclusion
  760. \end_layout
  761. \begin_layout Standard
  762. There are numerous machine learning algorithms available and suitable for
  763. use in activity classification tasks.
  764. There has been a lot of research into their use, and all of the algorithms
  765. discussed have produced good results.
  766. Because of the lack of need for any training, however, the K-Nearest Neighbour
  767. algorithm appears to be the most promising for a mobile device.
  768. Any algorithm that needs explicit training prior to classification would
  769. almost certainly require a desktop application or a remote service to analyse
  770. the data, as it typically requires large amounts of memory and expensive
  771. computations.
  772. This either makes the application extremely cumbersome for the user (they
  773. have to connect their phone to a computer, transfer a file, obtain and
  774. run a separate application, then transfer some file back), or puts a large
  775. resource burden onto the distributor (having to remotely analyse all of
  776. the data from all users would require dedicated hardware for any more than
  777. a few users).
  778. \end_layout
  779. \begin_layout Section
  780. Mobile telephones
  781. \end_layout
  782. \begin_layout Standard
  783. It's hard to overstate the ubiquity of mobile telephones at present.
  784. In 2003, over a billion mobile telephones were sold - six times as many
  785. as the number of personal computers
  786. \begin_inset CommandInset citation
  787. LatexCommand cite
  788. key "Eagle2004"
  789. \end_inset
  790. .
  791. In 2007, this same figure describes the number of cameraphones sold
  792. \begin_inset CommandInset citation
  793. LatexCommand cite
  794. key "Reynolds2008"
  795. \end_inset
  796. , clearly representing a substantial growth in sales and advancements in
  797. the technology.
  798. In fact, mobile telephones are the fastest adopted technology in human
  799. history
  800. \begin_inset CommandInset citation
  801. LatexCommand cite
  802. key "Eagle2004"
  803. \end_inset
  804. .
  805. This ubiquity, coupled with the fact that mobile telephones are comfortably
  806. carried around on a daily basis by most of their users, makes them a very
  807. attractive alternative to more traditional platforms used for activity-aware
  808. research, which typically involved bulky or inconvenient apparatus that
  809. was expensive to manufacture
  810. \begin_inset CommandInset citation
  811. LatexCommand cite
  812. key "Schmidt2008"
  813. \end_inset
  814. and made users very self-conscious.
  815. \end_layout
  816. \begin_layout Subsection
  817. iPhone
  818. \end_layout
  819. \begin_layout Standard
  820. There have been several published works related to activity-recognition
  821. on the iPhone.
  822. The similarity between iPhone and Android platforms means that many of
  823. the concepts developed for or used on the iPhone are applicable to both.
  824. \end_layout
  825. \begin_layout Subsubsection
  826. iLearn
  827. \end_layout
  828. \begin_layout Standard
  829. \noun on
  830. iLearn
  831. \noun default
  832. \begin_inset CommandInset citation
  833. LatexCommand cite
  834. key "Schmidt2008"
  835. \end_inset
  836. is a suite of three tools -
  837. \noun on
  838. iLog
  839. \noun default
  840. ,
  841. \noun on
  842. iModel,
  843. \noun default
  844. and
  845. \noun on
  846. iClassify
  847. \noun default
  848. - which together allow for real-time classification of low-level activities.
  849. \noun on
  850. iLog
  851. \noun default
  852. is run on the user's iPhone and allows the user to specify which activity
  853. they will be performing.
  854. The application then records raw sensor data from the iPhone's three-axis
  855. accelerometer and 124 features computed from this data in real-time.
  856. The data is then stored on the device, annotated with the selected activity.
  857. \end_layout
  858. \begin_layout Standard
  859. The training data collected by
  860. \noun on
  861. iLog
  862. \noun default
  863. is then transferred to a desktop computer where
  864. \noun on
  865. iModel
  866. \noun default
  867. uses a Naïve Bayesian Network (NBN) to create a model which can be used
  868. to classify future input.
  869. The choice of NBNs was based on their ability to classify a set of trial
  870. data correctly, and the low computational cost of classifying data once
  871. the model has been generated.
  872. \end_layout
  873. \begin_layout Standard
  874. Once the model has been created, it is transferred back to the device where
  875. it is used by
  876. \noun on
  877. iClassify
  878. \noun default
  879. .
  880. This provides an API for other applications, and allows them to register
  881. for a callback which it publishes the user's current activity to every
  882. second.
  883. \end_layout
  884. \begin_layout Standard
  885. Unfortunately, neither the source code nor the API are published.
  886. The inability to run background processes on the iPhone suggests that any
  887. \begin_inset Quotes eld
  888. \end_inset
  889. API
  890. \begin_inset Quotes erd
  891. \end_inset
  892. would have to be more like a framework where the third-party developer
  893. has to re-engineer their application to use the
  894. \noun on
  895. iClassify
  896. \noun default
  897. application as a base.
  898. This is undesirable as it makes it extremely difficult to adapt existing
  899. applications to use the activity-aware API, and is a very cumbersome way
  900. of providing what could be a very minor piece of functionality for the
  901. application.
  902. \end_layout
  903. \begin_layout Subsubsection
  904. Evaluation
  905. \end_layout
  906. \begin_layout Standard
  907. \begin_inset CommandInset citation
  908. LatexCommand citet
  909. key "Miluzzo2009"
  910. \end_inset
  911. present an evaluation of the iPhone for use in
  912. \begin_inset Quotes eld
  913. \end_inset
  914. people-centric sensing applications
  915. \begin_inset Quotes erd
  916. \end_inset
  917. .
  918. One of the major drawbacks highlighted is that the iPhone does not support
  919. applications which run in the background.
  920. This means that any application wishing to perform continuous real-time
  921. activity detection would need to run as a foreground process, preventing
  922. the user from using the device for any other function.
  923. \end_layout
  924. \begin_layout Standard
  925. The research also shows that the computational compatibility of the iPhone
  926. is more than sufficient to perform the necessary calculations for a typical
  927. activity-recognising application, which suggests that any modern smart
  928. phone would be capable of these.
  929. \end_layout
  930. \begin_layout Subsection
  931. Android
  932. \end_layout
  933. \begin_layout Standard
  934. While the Android platform is relatively new, it is rapidly gaining market
  935. share on the more established mobile operating systems.
  936. A December 2009 survey
  937. \begin_inset CommandInset citation
  938. LatexCommand cite
  939. key "ChangeWave2010"
  940. \end_inset
  941. shows that 21% of respondents want their next smartphone purchase to run
  942. Android, a 350% increase from the same survey conducted three months prior.
  943. This is compared to the iPhone, which dropped 4% to 28% in the same time
  944. period.
  945. Gartner, a respected IT research firm, predicts that by 2012, Android will
  946. be the second most popular mobile operating system globally
  947. \begin_inset CommandInset citation
  948. LatexCommand cite
  949. key "ComputerWorld2010"
  950. \end_inset
  951. .
  952. \end_layout
  953. \begin_layout Standard
  954. In addition to its rapidly increasing popularity, the Android platform offers
  955. several advantages over the iPhone platform.
  956. Most notably is the ability to run background processes (called
  957. \noun on
  958. services
  959. \noun default
  960. ), which will allow a classifier application to run without interfering
  961. with the user's normal use of their mobile telephone.
  962. In addition, the Android OS provides access to the Bluetooth and GSM stacks,
  963. allowing for data from both to be used for activity detection.
  964. \end_layout
  965. \begin_layout Standard
  966. The ability to run a background process will enable a proper API for sharing
  967. activity data with other applications, which will allow third-party developers
  968. to make their applications context-aware with relatively little work on
  969. their part.
  970. This is extremely desirable as it will allow rapid prototyping of applications,
  971. which will hopefully lead to innovative new uses of activity classification.
  972. \end_layout
  973. \begin_layout Standard
  974. While it is purported
  975. \begin_inset CommandInset citation
  976. LatexCommand cite
  977. key "Garakani2009"
  978. \end_inset
  979. that there is research being done on bringing activity-awareness to Android
  980. platforms, there does not seem to be any work published on this matter
  981. or any applications available to support it.
  982. While there a small number of self-proclaimed
  983. \begin_inset Quotes eld
  984. \end_inset
  985. context-aware
  986. \begin_inset Quotes erd
  987. \end_inset
  988. applications for Android, this context is almost exclusively limited to
  989. geolocation.
  990. This project will therefore produce one of the first publicly available
  991. activity-aware applications for the Android platform.
  992. \end_layout
  993. \begin_layout Section
  994. Location analysis
  995. \begin_inset CommandInset label
  996. LatexCommand label
  997. name "sec:Location-analysis"
  998. \end_inset
  999. \end_layout
  1000. \begin_layout Standard
  1001. Location-based services are currently undergoing an
  1002. \begin_inset Quotes eld
  1003. \end_inset
  1004. explosion
  1005. \begin_inset Quotes erd
  1006. \end_inset
  1007. \begin_inset CommandInset citation
  1008. LatexCommand cite
  1009. key "Bellavista2008"
  1010. \end_inset
  1011. , thanks to improvements in technology, and greater openness on the part
  1012. of service providers and handset manufacturers.
  1013. All modern smartphone platforms have a geolocation stack, usually backed
  1014. by a GPS chipset and in most cases augmented with either a database of
  1015. known cell tower locations, or a map of known wifi network identifiers
  1016. and locations, or both.
  1017. The two databases allow for rough geolocation when GPS is not available,
  1018. or for greatly decreased lookup time when a GPS lock is available.
  1019. \end_layout
  1020. \begin_layout Standard
  1021. However, while the geolocation stack is a rich source of data, it is a poor
  1022. source of information.
  1023. A latitude/longitude pair may describe the user's exact location, but a
  1024. user would be hard-pressed to tell the difference between the latitude/longitud
  1025. e of their home, place of work, or of somewhere in between the two with
  1026. no real significance.
  1027. A great deal of research has therefore been devoted to detecting meaningful
  1028. locations from GPS traces.
  1029. \end_layout
  1030. \begin_layout Standard
  1031. \begin_inset Quotes eld
  1032. \end_inset
  1033. Place recognition
  1034. \begin_inset Quotes erd
  1035. \end_inset
  1036. has two phases: learning and recognising.
  1037. An initial learning phase analyses a sensor log and segments the data into
  1038. places where the device is stable (stationary), and designates this as
  1039. a
  1040. \begin_inset Quotes eld
  1041. \end_inset
  1042. waypoint
  1043. \begin_inset Quotes erd
  1044. \end_inset
  1045. .
  1046. It then merges
  1047. \begin_inset Quotes eld
  1048. \end_inset
  1049. waypoints
  1050. \begin_inset Quotes erd
  1051. \end_inset
  1052. that appear to identify the same place being visited multiple times.
  1053. The second phase uses these learned waypoints to recognise when the device
  1054. is revisiting a place, and therefore also when the device is not visiting
  1055. a previously known place (for example when it is moving between two)
  1056. \begin_inset CommandInset citation
  1057. LatexCommand cite
  1058. key "Hightower2005"
  1059. \end_inset
  1060. .
  1061. \end_layout
  1062. \begin_layout Standard
  1063. Unfortunately, quite a lot of research into location analysis uses GPS
  1064. \begin_inset Quotes eld
  1065. \end_inset
  1066. blackspots
  1067. \begin_inset Quotes erd
  1068. \end_inset
  1069. to identify useful places
  1070. \begin_inset CommandInset citation
  1071. LatexCommand cite
  1072. key "Nurmi2006,Liao2007b"
  1073. \end_inset
  1074. .
  1075. With older GPS chipsets, the satellite signal would be lost when the user
  1076. entered a building, and this allowed an inference that the current location
  1077. was probably a place of interest.
  1078. However, modern GPS chipsets receive a signal in most indoor locations.
  1079. It is possible that a decrease in signal strength or number of locked satellite
  1080. s may still occur, or that GSM signal strength could be used instead, but
  1081. these ideas have not been widely explored at present.
  1082. \end_layout
  1083. \begin_layout Standard
  1084. There is, however, plenty of research relating to the use of location data
  1085. outdoors.
  1086. One application
  1087. \begin_inset CommandInset citation
  1088. LatexCommand cite
  1089. key "Liao2007b"
  1090. \end_inset
  1091. learns not only the user's frequently visited places, but the method of
  1092. transport used between them and the typical routes taken.
  1093. It can then offer instructions showing the user how to go from place to
  1094. place, or issue alerts if the user appears to be going the wrong way (by
  1095. getting on the wrong bus, for instance).
  1096. The ability to correctly infer the user's destination would be extremely
  1097. useful in a context-aware system: a user walking to do their grocery shopping
  1098. is almost certainly going to want to interact with their phone differently
  1099. than a user on a bus going to work.
  1100. \end_layout
  1101. \begin_layout Section
  1102. Bluetooth
  1103. \begin_inset CommandInset label
  1104. LatexCommand label
  1105. name "sec:Bluetooth"
  1106. \end_inset
  1107. \end_layout
  1108. \begin_layout Standard
  1109. The user's context depends on not only what they are doing, where they are
  1110. doing it, but also who they are with.
  1111. Sitting and eating lunch with a manager is quite a different context to
  1112. sitting and eating lunch with a spouse.
  1113. It would therefore be desirable to be able to identify between different
  1114. people when performing context analysis.
  1115. \end_layout
  1116. \begin_layout Standard
  1117. One of the few ways that a mobile telephone can identify other people is
  1118. by searching for
  1119. \emph on
  1120. their
  1121. \emph default
  1122. mobile telephones.
  1123. This can be done by scanning for Bluetooth devices, which involves broadcasting
  1124. a
  1125. \begin_inset Quotes eld
  1126. \end_inset
  1127. device inquiry
  1128. \begin_inset Quotes erd
  1129. \end_inset
  1130. message; if a device chooses to answer the inquiry, it discloses its unique
  1131. MAC address and device class
  1132. \begin_inset Foot
  1133. status collapsed
  1134. \begin_layout Plain Layout
  1135. the device class tells us whether the device is a computer or a mobile telephone
  1136. , for example
  1137. \end_layout
  1138. \end_inset
  1139. .
  1140. Unfortunately, this requires the person to not only be carrying a mobile
  1141. telephone, but a Bluetooth-enabled model, and for them to have configured
  1142. their device to have Bluetooth enabled and to be
  1143. \begin_inset Quotes eld
  1144. \end_inset
  1145. visible
  1146. \begin_inset Quotes erd
  1147. \end_inset
  1148. .
  1149. A study in 2004
  1150. \begin_inset CommandInset citation
  1151. LatexCommand cite
  1152. key "Eagle2004"
  1153. \end_inset
  1154. found that only 1 in 150 people had such a configured device on a university
  1155. campus.
  1156. This figure will undoubtedly be greater now, and may well be greater when
  1157. in public, but it highlights that only a handful of people may be detectable
  1158. via their Bluetooth devices.
  1159. \end_layout
  1160. \begin_layout Standard
  1161. A study in 2006
  1162. \begin_inset CommandInset citation
  1163. LatexCommand cite
  1164. key "Nicolai2006"
  1165. \end_inset
  1166. used a similar technique to monitor the social context of the user, introducing
  1167. the idea of
  1168. \begin_inset Quotes eld
  1169. \end_inset
  1170. familiar
  1171. \begin_inset Quotes erd
  1172. \end_inset
  1173. people,
  1174. \begin_inset Quotes eld
  1175. \end_inset
  1176. unfamiliar
  1177. \begin_inset Quotes erd
  1178. \end_inset
  1179. people and
  1180. \begin_inset Quotes eld
  1181. \end_inset
  1182. familiar strangers
  1183. \begin_inset Quotes erd
  1184. \end_inset
  1185. .
  1186. These labels were applied based on the number of times their Bluetooth
  1187. devices were detected
  1188. \begin_inset Foot
  1189. status collapsed
  1190. \begin_layout Plain Layout
  1191. and by extension the number of times the user had come into contact with
  1192. them
  1193. \end_layout
  1194. \end_inset
  1195. .
  1196. While the definition of
  1197. \begin_inset Quotes eld
  1198. \end_inset
  1199. familiar
  1200. \begin_inset Quotes erd
  1201. \end_inset
  1202. and
  1203. \begin_inset Quotes eld
  1204. \end_inset
  1205. unfamiliar
  1206. \begin_inset Quotes erd
  1207. \end_inset
  1208. are quite obvious,
  1209. \begin_inset Quotes eld
  1210. \end_inset
  1211. familiar strangers
  1212. \begin_inset Quotes erd
  1213. \end_inset
  1214. is a new class of people used to describe those who the user encounters
  1215. repeatedly, but doesn't interact with.
  1216. This may include neighbours that are passed on the street, or fellow commuters
  1217. on a journey into work.
  1218. The number of people in each of those groups (and any changes in those
  1219. numbers) can be used to infer how
  1220. \begin_inset Quotes eld
  1221. \end_inset
  1222. comfortable
  1223. \begin_inset Quotes erd
  1224. \end_inset
  1225. the user feels with their social context, and whether their current activity
  1226. is part of a normal routine or is novel.
  1227. \end_layout
  1228. \begin_layout Standard
  1229. This research has, to date, not been readily combined with activity-aware
  1230. applications, and this project will aim to integrate the results of Bluetooth
  1231. scanning with
  1232. \begin_inset Quotes eld
  1233. \end_inset
  1234. classical
  1235. \begin_inset Quotes erd
  1236. \end_inset
  1237. activity classification techniques and to evaluate whether it provides
  1238. any benefit.
  1239. \end_layout
  1240. \begin_layout Section
  1241. Power management
  1242. \begin_inset CommandInset label
  1243. LatexCommand label
  1244. name "sec:Power-management"
  1245. \end_inset
  1246. \end_layout
  1247. \begin_layout Standard
  1248. One major consideration when deploying an application on a mobile device
  1249. is the amount of power it will use.
  1250. An application constantly polling any one sensor can reduce battery life
  1251. significantly, and an application which kept all available sensors active
  1252. (in addition to doing CPU-heavy analysis on them) would drain the battery
  1253. in a typical smartphone in a matter of hours.
  1254. A context-aware application is not very useful for a user if they can only
  1255. use their telephone for an hour or two before it needs recharging!
  1256. \end_layout
  1257. \begin_layout Standard
  1258. One solution
  1259. \begin_inset CommandInset citation
  1260. LatexCommand cite
  1261. key "Wang2009"
  1262. \end_inset
  1263. is to only use one or two sensors to monitor the user's activity until it
  1264. appears to be transitioning.
  1265. For example, if the user is believed to be walking, the application only
  1266. needs to periodically check either the accelerometer (to confirm the user
  1267. is still making walking motions) or GPS (to confirm the distance traveled
  1268. is still consistent with walking) to know that their activity has not changed.
  1269. As soon as the user's behaviour becomes inconsistent with walking, the
  1270. application can bring other sensors online until it has successfully reclassifi
  1271. ed the activity, and then resume monitoring with minimal sensors.
  1272. \end_layout
  1273. \begin_layout Standard
  1274. Another option
  1275. \begin_inset CommandInset citation
  1276. LatexCommand cite
  1277. key "Wang2009"
  1278. \end_inset
  1279. (which can be used in conjunction) is to only enable sensors for a short
  1280. amount of time, and then sleep for a period before reactivating them.
  1281. The
  1282. \begin_inset Quotes eld
  1283. \end_inset
  1284. duty cycle
  1285. \begin_inset Quotes erd
  1286. \end_inset
  1287. suggested for accelerometers is 6 second of sensing followed by 10 seconds
  1288. of sleeping.
  1289. The six second window is enough time to allow for capturing a full range
  1290. of motion (several complete strides) if the user is walking or running,
  1291. and then the ten second sleep stops the accelerometer using battery power
  1292. until the next cycle.
  1293. This process obviously means that a sudden switch in activity will not
  1294. be noticed immediately, but a delay of a few seconds is acceptable as most
  1295. activities will last for minutes or longer.
  1296. \end_layout
  1297. \begin_layout Standard
  1298. The battery life on modern smartphones rarely exceeds 24 hours of typical
  1299. use, so it is extremely important that any applications developed for this
  1300. project does not significantly reduce this.
  1301. A balance between prompt detection and notification of activity changes
  1302. and battery use by sensors and processing algorithms will need to be found.
  1303. \end_layout
  1304. \begin_layout Standard
  1305. \begin_inset Newpage pagebreak
  1306. \end_inset
  1307. \end_layout
  1308. \begin_layout Part
  1309. Specification
  1310. \end_layout
  1311. \begin_layout Section
  1312. Experimentation
  1313. \begin_inset CommandInset label
  1314. LatexCommand label
  1315. name "sec:Experimentation"
  1316. \end_inset
  1317. \end_layout
  1318. \begin_layout Subsection
  1319. Battery use
  1320. \end_layout
  1321. \begin_layout Standard
  1322. The average lifetime of a typical Android mobile telephone will need to
  1323. be ascertained, taking into consideration typical day-to-day use of the
  1324. device and any additional battery drain caused by that.
  1325. The impact of sensor use on the battery lifetime needs to be calculated
  1326. or experimentally determined, and the effect of the algorithms described
  1327. in section
  1328. \begin_inset CommandInset ref
  1329. LatexCommand ref
  1330. reference "sec:Power-management"
  1331. \end_inset
  1332. needs to be quantified.
  1333. \end_layout
  1334. \begin_layout Standard
  1335. Once this is complete, a selection of algorithms and suitable parameters
  1336. should be made for use in the classifier application (section
  1337. \begin_inset CommandInset ref
  1338. LatexCommand ref
  1339. reference "sub:Classifier-application"
  1340. \end_inset
  1341. ).
  1342. The aim of the selection should be to ensure the best activity detection
  1343. whilst not severely reducing the device's battery life.
  1344. \end_layout
  1345. \begin_layout Subsection
  1346. Location analysis
  1347. \begin_inset CommandInset label
  1348. LatexCommand label
  1349. name "sub:Location-analysis"
  1350. \end_inset
  1351. \end_layout
  1352. \begin_layout Standard
  1353. As discussed in section
  1354. \begin_inset CommandInset ref
  1355. LatexCommand ref
  1356. reference "sec:Location-analysis"
  1357. \end_inset
  1358. , existing algorithms to detect interesting places rely on GPS signals not
  1359. being obtainable indoors, which is no longer true with current generation
  1360. GPS chipsets.
  1361. It may be possible to adapt these algorithms to work instead using the
  1362. GSM signal strength or number of locked GPS satellites.
  1363. The variance of these when inside and outside a building need to be investigate
  1364. d to determine if this is a feasible approach, or whether an alternate algorithm
  1365. needs to be used.
  1366. \end_layout
  1367. \begin_layout Standard
  1368. A suitable algorithm should be implemented which allows the user's activity
  1369. to be annotated with location-based context.
  1370. This would ideally take the form of a source and destination for mobile
  1371. activities, or location for static activities.
  1372. The goal is to be able to differentiate between
  1373. \begin_inset Quotes eld
  1374. \end_inset
  1375. walking to work
  1376. \begin_inset Quotes erd
  1377. \end_inset
  1378. and
  1379. \begin_inset Quotes eld
  1380. \end_inset
  1381. walking to the shop
  1382. \begin_inset Quotes erd
  1383. \end_inset
  1384. , although the labeling of places may be left to future work.
  1385. Unlabeled places (i.e., something akin to
  1386. \begin_inset Quotes eld
  1387. \end_inset
  1388. walking to place2
  1389. \begin_inset Quotes erd
  1390. \end_inset
  1391. and
  1392. \begin_inset Quotes eld
  1393. \end_inset
  1394. walking to place7
  1395. \begin_inset Quotes erd
  1396. \end_inset
  1397. instead of the previous examples) still allow for applications to tailor
  1398. their behaviour based on the user's previous activities at or when walking
  1399. to that place, so will be sufficient for an initial version.
  1400. \end_layout
  1401. \begin_layout Subsection
  1402. Bluetooth usage
  1403. \begin_inset CommandInset label
  1404. LatexCommand label
  1405. name "sub:Bluetooth-usage"
  1406. \end_inset
  1407. \end_layout
  1408. \begin_layout Standard
  1409. The algorithm for classifying familiar, unfamiliar, and
  1410. \begin_inset Quotes eld
  1411. \end_inset
  1412. familiar stranger
  1413. \begin_inset Quotes erd
  1414. \end_inset
  1415. Bluetooth devices (section
  1416. \begin_inset CommandInset ref
  1417. LatexCommand ref
  1418. reference "sec:Bluetooth"
  1419. \end_inset
  1420. ) should be implemented.
  1421. An experiment should then be conducted to determine whether or not the
  1422. data provides useful clues as to the user's context.
  1423. This will depend on the number of other people with Bluetooth discoverable
  1424. devices in the vicinity of the user.
  1425. \end_layout
  1426. \begin_layout Standard
  1427. If there are a sufficient number of discoverable devices, then the algorithms
  1428. should be included in the classifier application to provide additional
  1429. context.
  1430. \end_layout
  1431. \begin_layout Section
  1432. Deliverables
  1433. \begin_inset CommandInset label
  1434. LatexCommand label
  1435. name "sec:Deliverables"
  1436. \end_inset
  1437. \end_layout
  1438. \begin_layout Standard
  1439. The following items should be delivered:
  1440. \end_layout
  1441. \begin_layout Subsection
  1442. Context-aware Framework
  1443. \begin_inset CommandInset label
  1444. LatexCommand label
  1445. name "sub:Classifier-application"
  1446. \end_inset
  1447. \end_layout
  1448. \begin_layout Standard
  1449. An Android application which:
  1450. \end_layout
  1451. \begin_layout Itemize
  1452. monitors raw sensor data and extracts features from the device's:
  1453. \end_layout
  1454. \begin_deeper
  1455. \begin_layout Itemize
  1456. accelerometer sensors
  1457. \end_layout
  1458. \begin_layout Itemize
  1459. magnetic field sensors
  1460. \end_layout
  1461. \begin_layout Itemize
  1462. camera
  1463. \end_layout
  1464. \begin_layout Itemize
  1465. microphone
  1466. \end_layout
  1467. \end_deeper
  1468. \begin_layout Itemize
  1469. monitors relevant data and extracts features concerning:
  1470. \end_layout
  1471. \begin_deeper
  1472. \begin_layout Itemize
  1473. visible Bluetooth devices in proximity to the device
  1474. \end_layout
  1475. \begin_layout Itemize
  1476. the current location of the device
  1477. \end_layout
  1478. \begin_layout Itemize
  1479. the current GSM signal strength and cell ID
  1480. \end_layout
  1481. \end_deeper
  1482. \begin_layout Itemize
  1483. presents the user with a method of optionally annotating this data with
  1484. their current activity
  1485. \end_layout
  1486. \begin_layout Itemize
  1487. classifies the user's current activity using the extracted features and
  1488. a K-Nearest Neighbour algorithm
  1489. \end_layout
  1490. \begin_layout Itemize
  1491. enhances the classification with extra contextual data inferred from location
  1492. or Bluetooth service:
  1493. \end_layout
  1494. \begin_deeper
  1495. \begin_layout Itemize
  1496. the user's probable source and destination, if traveling
  1497. \end_layout
  1498. \begin_layout Itemize
  1499. the user's current location, if stationary
  1500. \end_layout
  1501. \begin_layout Itemize
  1502. the user's company, if detected
  1503. \end_layout
  1504. \end_deeper
  1505. \begin_layout Itemize
  1506. attempts to conserve battery life by intelligent management and timing of
  1507. sensor activity
  1508. \end_layout
  1509. \begin_layout Subsubsection
  1510. Interface
  1511. \end_layout
  1512. \begin_layout Standard
  1513. The application should provide two different interfaces to retrieve the
  1514. user's context.
  1515. The first should be an implementation of the Android ContentProvider interface.
  1516. This allows third party applications to query and retrieve data (in this
  1517. case, the user's probable activities and context) as and when they need
  1518. it.
  1519. \end_layout
  1520. \begin_layout Standard
  1521. The ContentProvider interface exposes data as a table.
  1522. The classifier application should therefore supply a unique ID for each
  1523. possible activity, the name of the activity (if labeled), the time the
  1524. activity was started, and the estimated uncertainty in the classification.
  1525. It should also expose relevant details of any context information, such
  1526. as related places (s
  1527. \begin_inset CommandInset ref
  1528. LatexCommand ref
  1529. reference "sub:Location-analysis"
  1530. \end_inset
  1531. ) or other participants (s
  1532. \begin_inset CommandInset ref
  1533. LatexCommand ref
  1534. reference "sub:Bluetooth-usage"
  1535. \end_inset
  1536. ).
  1537. \end_layout
  1538. \begin_layout Standard
  1539. The second interface should use Android broadcast Intents to notify any
  1540. interested application whenever the user's activity changes.
  1541. This will allow applications to be notified whenever the activity or context
  1542. of the user changes.
  1543. The Intent should contain a URI which corresponds to an entry in the table
  1544. returned by the ContentProvider, so applications can retrieve information
  1545. about the user's activity directly.
  1546. \end_layout
  1547. \begin_layout Standard
  1548. Whenever activities, places or participants are exposed in the interface,
  1549. they should be assigned an ID which remains consistent for the specified
  1550. activity, place or participant.
  1551. Applications can use this ID regardless of any annotations the user supplies.
  1552. For example, the first time an interesting place is detected it may be
  1553. assigned ID 42 and label
  1554. \begin_inset Quotes eld
  1555. \end_inset
  1556. place42
  1557. \begin_inset Quotes erd
  1558. \end_inset
  1559. , the user may then annotate this place as
  1560. \begin_inset Quotes eld
  1561. \end_inset
  1562. home
  1563. \begin_inset Quotes erd
  1564. \end_inset
  1565. , and some time later replace it with
  1566. \begin_inset Quotes eld
  1567. \end_inset
  1568. old house
  1569. \begin_inset Quotes erd
  1570. \end_inset
  1571. ; the ID will be consistent across all of these labels, so any application
  1572. using this information will continue to function.
  1573. \end_layout
  1574. \begin_layout Subsection
  1575. Activity condition for Locale
  1576. \end_layout
  1577. \begin_layout Standard
  1578. As a proof-of-concept, an application should be developed which provides
  1579. Locale
  1580. \begin_inset CommandInset citation
  1581. LatexCommand cite
  1582. key "TwoFortyFourAm2010"
  1583. \end_inset
  1584. , a popular Android application used to alter device settings and perform
  1585. actions based on certain conditions, with a new condition representing
  1586. the user's activity.
  1587. \end_layout
  1588. \begin_layout Standard
  1589. This will allow users to change their device settings based on their current
  1590. activity by creating rules within Locale.
  1591. Locale also allows users to post updates to popular social networking sites
  1592. such as Twitter and Facebook, so it will be possible for users to broadcast
  1593. their activities to their friends.
  1594. \end_layout
  1595. \begin_layout Subsection
  1596. Context-aware home screen
  1597. \end_layout
  1598. \begin_layout Standard
  1599. To demonstrate the utility of applications being context-aware, an application
  1600. will be developed that replaces the standard Android home screen.
  1601. It should dynamically adapt its layout and contents according to the user's
  1602. activity and other context information made available by the classifier
  1603. application.
  1604. \end_layout
  1605. \begin_layout Standard
  1606. The replacement home screen needs to allow users to browse and launch applicatio
  1607. ns, and should expose relevant (context-sensitive) notifications to the
  1608. user, including notifications of e-mails, missed calls, text messages and
  1609. upcoming appointments.
  1610. \end_layout
  1611. \begin_layout Subsection
  1612. User and developer guides
  1613. \end_layout
  1614. \begin_layout Itemize
  1615. A user guide for the classifier
  1616. \noun on
  1617. \noun default
  1618. application
  1619. \end_layout
  1620. \begin_layout Itemize
  1621. A user guide for the Locale condition provider
  1622. \end_layout
  1623. \begin_layout Itemize
  1624. A user guide for the Context-aware home screen
  1625. \end_layout
  1626. \begin_layout Itemize
  1627. A developer's guide for using the information exposed by the
  1628. \noun on
  1629. classifier
  1630. \noun default
  1631. application
  1632. \end_layout
  1633. \begin_layout Subsection
  1634. Website
  1635. \end_layout
  1636. \begin_layout Standard
  1637. A website should be created containing an overview of the applications,
  1638. instructions for both users and developers, and links to download the applicati
  1639. ons.
  1640. The website should explain the basic ideas behind activity and context
  1641. aware applications, and the specific techniques used in this project.
  1642. \end_layout
  1643. \begin_layout Standard
  1644. \begin_inset Newpage pagebreak
  1645. \end_inset
  1646. \end_layout
  1647. \begin_layout Part
  1648. Evaluation
  1649. \end_layout
  1650. \begin_layout Section
  1651. Reports
  1652. \end_layout
  1653. \begin_layout Standard
  1654. The results of the experimentation described in section
  1655. \begin_inset CommandInset ref
  1656. LatexCommand ref
  1657. reference "sec:Experimentation"
  1658. \end_inset
  1659. should be written up as a report.
  1660. The reports must include the data collected in each of the experiments,
  1661. the conclusions drawn from those, and the impact of the results of the
  1662. experiment on the project deliverables.
  1663. \end_layout
  1664. \begin_layout Section
  1665. Deliverables
  1666. \end_layout
  1667. \begin_layout Standard
  1668. On successful completion of the project, there should be three deliverable
  1669. applications as specified in section
  1670. \begin_inset CommandInset ref
  1671. LatexCommand ref
  1672. reference "sec:Deliverables"
  1673. \end_inset
  1674. .
  1675. These can be tested and evaluated in a variety of manners:
  1676. \end_layout
  1677. \begin_layout Subsection
  1678. Unit tests
  1679. \end_layout
  1680. \begin_layout Standard
  1681. Throughout the development of the project, unit tests should be created
  1682. to test key functionality of all applications.
  1683. It is expected that at the completion of the project, all unit tests should
  1684. pass successfully, and they will have a code coverage of 80% or above.
  1685. \end_layout
  1686. \begin_layout Subsection
  1687. System tests
  1688. \end_layout
  1689. \begin_layout Standard
  1690. The classifier application should also have a suite of system tests.
  1691. These should consist of a set of fake or pre-recorded inputs which are
  1692. fed into the application in place of raw sensor data.
  1693. The output of the classifier (via the API) can then be compared to expected
  1694. output for the data.
  1695. \end_layout
  1696. \begin_layout Subsection
  1697. User acceptance testing
  1698. \end_layout
  1699. \begin_layout Standard
  1700. The Locale addon and context-aware home screen should be subject to user
  1701. acceptance testing for evaluation.
  1702. This should take the form of providing the applications to multiple end
  1703. users, allowing them to use them for a period of time (providing instructions
  1704. for certain tasks to complete).
  1705. The users should then be presented with a questionnaire which they can
  1706. use to evaluate the functionality, utility and design of the applications.
  1707. \end_layout
  1708. \begin_layout Subsubsection
  1709. Android market
  1710. \end_layout
  1711. \begin_layout Standard
  1712. In addition to providing the applications to a closed set of users, the
  1713. applications should be published to the Android market.
  1714. This will allow any owner of a compatible Android device to download and
  1715. use the applications.
  1716. The market also allows users to rate the application and leave comments,
  1717. which will be an extremely useful method of evaluating the success of the
  1718. applications.
  1719. \end_layout
  1720. \begin_layout Standard
  1721. As part of this, the classifier application/framework will be published,
  1722. and should include instructions for developers detailing how they can make
  1723. their applications context-aware.
  1724. The interest expressed and number of applications which use it (if any)
  1725. will be an indicator of the effectiveness of the API and its documentation.
  1726. \end_layout
  1727. \begin_layout Subsection
  1728. Classification scope
  1729. \end_layout
  1730. \begin_layout Standard
  1731. The classifier should be able to classify the following activities:
  1732. \end_layout
  1733. \begin_layout Itemize
  1734. walking
  1735. \end_layout
  1736. \begin_layout Itemize
  1737. running
  1738. \end_layout
  1739. \begin_layout Itemize
  1740. standing
  1741. \end_layout
  1742. \begin_layout Itemize
  1743. sitting
  1744. \end_layout
  1745. \begin_layout Itemize
  1746. traveling in a vehicle
  1747. \end_layout
  1748. \begin_layout Standard
  1749. These should be evaluated by means of a leave-one-out cross-validation test,
  1750. with data collected from one or more users and annotated by hand.
  1751. It is expected that the classifier should correctly classify all activities
  1752. with an accuracy of at least 70%, within 30 seconds of the activity being
  1753. started.
  1754. \end_layout
  1755. \begin_layout Subsection
  1756. Resource usage
  1757. \end_layout
  1758. \begin_layout Standard
  1759. Finally, the applications should be evaluated based on their resource usage.
  1760. One of the main concerns will be battery usage (caused both by the application
  1761. using processor time, and activating sensors), but attention must also
  1762. be paid to memory consumption (especially after extended use).
  1763. The goal should be to ensure that the resource usage of the applications
  1764. in this project do not adversely impact the functionality of the device;
  1765. that is, battery life should not be reduced so significantly that it requires
  1766. users to change their normal charging behaviour, and memory usage should
  1767. not be so high as to cause other applications to become sluggish or be
  1768. closed by the operating system.
  1769. \begin_inset Newpage pagebreak
  1770. \end_inset
  1771. \end_layout
  1772. \begin_layout Standard
  1773. \begin_inset CommandInset bibtex
  1774. LatexCommand bibtex
  1775. bibfiles "/home/chris/Uni/project/papers/project"
  1776. options "bibtotoc,savetrees"
  1777. \end_inset
  1778. \end_layout
  1779. \end_body
  1780. \end_document