|
@@ -1757,6 +1757,389 @@ The battery life on modern smartphones rarely exceeds 24 hours of typical
|
1757
|
1757
|
Accelerometer classification
|
1758
|
1758
|
\end_layout
|
1759
|
1759
|
|
|
1760
|
+\begin_layout Section
|
|
1761
|
+Sensor Logger application
|
|
1762
|
+\end_layout
|
|
1763
|
+
|
|
1764
|
+\begin_layout Standard
|
|
1765
|
+The primary component of context that this project aims to expose is the
|
|
1766
|
+ user's activity.
|
|
1767
|
+ The main factor in determining a user's activity is the data retrieved
|
|
1768
|
+ from the device's accelerometers.
|
|
1769
|
+ This gave rise to the project's first published application, titled
|
|
1770
|
+\noun on
|
|
1771
|
+Sensor Logger
|
|
1772
|
+\noun default
|
|
1773
|
+.
|
|
1774
|
+ The first version of
|
|
1775
|
+\noun on
|
|
1776
|
+Sensor Logger
|
|
1777
|
+\noun default
|
|
1778
|
+ consisted of a single activity containing a large amount of text describing
|
|
1779
|
+ the project, an editable text field where the user could name their activity,
|
|
1780
|
+ and a button which initiated logging.
|
|
1781
|
+\end_layout
|
|
1782
|
+
|
|
1783
|
+\begin_layout Standard
|
|
1784
|
+Once the user tapped the 'Start' button, the application launched a service
|
|
1785
|
+ which registered with the device's SensorManager and requested fast updates
|
|
1786
|
+ from both the accelerometer and the magnetic field sensor.
|
|
1787
|
+ Every 50ms the last value received from each axis on each sensor was written
|
|
1788
|
+ along with a timestamp to a file on the device.
|
|
1789
|
+ After 1024 samples were collected (a total of around 51 seconds), the service
|
|
1790
|
+ launched an uploader service and terminated itself.
|
|
1791
|
+ The uploader service read the file from the device's flash memory, opened
|
|
1792
|
+ a HTTP connection to the project website, and submitted the data to a PHP
|
|
1793
|
+ script.
|
|
1794
|
+ The PHP script in turn stored the data in a MySQL database.
|
|
1795
|
+\end_layout
|
|
1796
|
+
|
|
1797
|
+\begin_layout Standard
|
|
1798
|
+\begin_inset Note Note
|
|
1799
|
+status open
|
|
1800
|
+
|
|
1801
|
+\begin_layout Plain Layout
|
|
1802
|
+Market
|
|
1803
|
+\end_layout
|
|
1804
|
+
|
|
1805
|
+\end_inset
|
|
1806
|
+
|
|
1807
|
+
|
|
1808
|
+\end_layout
|
|
1809
|
+
|
|
1810
|
+\begin_layout Standard
|
|
1811
|
+\begin_inset Note Note
|
|
1812
|
+status open
|
|
1813
|
+
|
|
1814
|
+\begin_layout Plain Layout
|
|
1815
|
+Diagram of components
|
|
1816
|
+\end_layout
|
|
1817
|
+
|
|
1818
|
+\end_inset
|
|
1819
|
+
|
|
1820
|
+
|
|
1821
|
+\end_layout
|
|
1822
|
+
|
|
1823
|
+\begin_layout Section
|
|
1824
|
+Manual classification
|
|
1825
|
+\end_layout
|
|
1826
|
+
|
|
1827
|
+\begin_layout Standard
|
|
1828
|
+\begin_inset Note Note
|
|
1829
|
+status open
|
|
1830
|
+
|
|
1831
|
+\begin_layout Plain Layout
|
|
1832
|
+Screenshot of window classifications
|
|
1833
|
+\end_layout
|
|
1834
|
+
|
|
1835
|
+\end_inset
|
|
1836
|
+
|
|
1837
|
+
|
|
1838
|
+\end_layout
|
|
1839
|
+
|
|
1840
|
+\begin_layout Standard
|
|
1841
|
+Once the sensor data was logged in the database, a web interface provided
|
|
1842
|
+ a graphical representation of the acceleration and magnetic field readings.
|
|
1843
|
+ It also allowed a handful of authenticated users to manually classify overlappi
|
|
1844
|
+ng
|
|
1845
|
+\begin_inset Quotes eld
|
|
1846
|
+\end_inset
|
|
1847
|
+
|
|
1848
|
+windows
|
|
1849
|
+\begin_inset Quotes erd
|
|
1850
|
+\end_inset
|
|
1851
|
+
|
|
1852
|
+ of 128 data points.
|
|
1853
|
+ These users could define activities in a hierarchical fashion, starting
|
|
1854
|
+ with two root nodes - 'CLASSIFIED' and 'UNCLASSIFIED'.
|
|
1855
|
+ Early data submitted resulted in a classification hierarchy of:
|
|
1856
|
+\end_layout
|
|
1857
|
+
|
|
1858
|
+\begin_layout Itemize
|
|
1859
|
+UNCLASSIFIED
|
|
1860
|
+\end_layout
|
|
1861
|
+
|
|
1862
|
+\begin_deeper
|
|
1863
|
+\begin_layout Itemize
|
|
1864
|
+PENDING (not yet manually assigned)
|
|
1865
|
+\end_layout
|
|
1866
|
+
|
|
1867
|
+\begin_layout Itemize
|
|
1868
|
+UNKNOWN (unable to determine actual activity from user label)
|
|
1869
|
+\end_layout
|
|
1870
|
+
|
|
1871
|
+\begin_layout Itemize
|
|
1872
|
+DNI (short for
|
|
1873
|
+\begin_inset Quotes eld
|
|
1874
|
+\end_inset
|
|
1875
|
+
|
|
1876
|
+Do Not Include
|
|
1877
|
+\begin_inset Quotes erd
|
|
1878
|
+\end_inset
|
|
1879
|
+
|
|
1880
|
+, for windows which appear erroneous)
|
|
1881
|
+\end_layout
|
|
1882
|
+
|
|
1883
|
+\end_deeper
|
|
1884
|
+\begin_layout Itemize
|
|
1885
|
+CLASSIFIED
|
|
1886
|
+\end_layout
|
|
1887
|
+
|
|
1888
|
+\begin_deeper
|
|
1889
|
+\begin_layout Itemize
|
|
1890
|
+WALKING
|
|
1891
|
+\end_layout
|
|
1892
|
+
|
|
1893
|
+\begin_deeper
|
|
1894
|
+\begin_layout Itemize
|
|
1895
|
+STAIRS (not used in itself)
|
|
1896
|
+\end_layout
|
|
1897
|
+
|
|
1898
|
+\begin_deeper
|
|
1899
|
+\begin_layout Itemize
|
|
1900
|
+UP
|
|
1901
|
+\end_layout
|
|
1902
|
+
|
|
1903
|
+\begin_layout Itemize
|
|
1904
|
+DOWN
|
|
1905
|
+\end_layout
|
|
1906
|
+
|
|
1907
|
+\end_deeper
|
|
1908
|
+\end_deeper
|
|
1909
|
+\begin_layout Itemize
|
|
1910
|
+IDLE (not used in itself)
|
|
1911
|
+\end_layout
|
|
1912
|
+
|
|
1913
|
+\begin_deeper
|
|
1914
|
+\begin_layout Itemize
|
|
1915
|
+SITTING
|
|
1916
|
+\end_layout
|
|
1917
|
+
|
|
1918
|
+\begin_layout Itemize
|
|
1919
|
+STANDING
|
|
1920
|
+\end_layout
|
|
1921
|
+
|
|
1922
|
+\end_deeper
|
|
1923
|
+\begin_layout Itemize
|
|
1924
|
+VEHICLE (not used in itself)
|
|
1925
|
+\end_layout
|
|
1926
|
+
|
|
1927
|
+\begin_deeper
|
|
1928
|
+\begin_layout Itemize
|
|
1929
|
+CAR
|
|
1930
|
+\end_layout
|
|
1931
|
+
|
|
1932
|
+\begin_layout Itemize
|
|
1933
|
+BUS
|
|
1934
|
+\end_layout
|
|
1935
|
+
|
|
1936
|
+\end_deeper
|
|
1937
|
+\begin_layout Itemize
|
|
1938
|
+DANCING
|
|
1939
|
+\end_layout
|
|
1940
|
+
|
|
1941
|
+\end_deeper
|
|
1942
|
+\begin_layout Standard
|
|
1943
|
+It was observed that many samples had erroneous data either at the start
|
|
1944
|
+ or the end of the recording.
|
|
1945
|
+ This was a result of the user putting the device away in a pocket (as instructe
|
|
1946
|
+d) or picking it back up to check the results.
|
|
1947
|
+ A classification of 'DNI', short for 'Do Not Include' was therefore introduced
|
|
1948
|
+ which allowed the erroneous windows to be summarilary excluded from later
|
|
1949
|
+ analysis.
|
|
1950
|
+ Later iterations of the Sensor Logger application included a 10 second
|
|
1951
|
+ delay at the start to give the user time to put the device away, and sounded
|
|
1952
|
+ an audible alert at the end, which reduced the number of records submitted
|
|
1953
|
+ with bad data.
|
|
1954
|
+\end_layout
|
|
1955
|
+
|
|
1956
|
+\begin_layout Standard
|
|
1957
|
+Another problem with the user submitted data was that some activity descriptions
|
|
1958
|
+ didn't make sense.
|
|
1959
|
+ With the earliest versions of the Sensor Logger, a significant number of
|
|
1960
|
+ users entered their own name or a non-sensical string into the textbox
|
|
1961
|
+ labelled
|
|
1962
|
+\begin_inset Quotes eld
|
|
1963
|
+\end_inset
|
|
1964
|
+
|
|
1965
|
+Activity name
|
|
1966
|
+\begin_inset Quotes erd
|
|
1967
|
+\end_inset
|
|
1968
|
+
|
|
1969
|
+.
|
|
1970
|
+ This suggested that users were either not reading or misunderstanding the
|
|
1971
|
+ instructions.
|
|
1972
|
+ A classification for 'UNKNOWN' was introduced to facilitate removal of
|
|
1973
|
+ the records where classifications couldn't be inferred from the activity
|
|
1974
|
+ name.
|
|
1975
|
+\end_layout
|
|
1976
|
+
|
|
1977
|
+\begin_layout Section
|
|
1978
|
+Feature extraction
|
|
1979
|
+\end_layout
|
|
1980
|
+
|
|
1981
|
+\begin_layout Standard
|
|
1982
|
+Once a reasonable sample of data had been recorded, the PHP script was modified
|
|
1983
|
+ to allow exporting of all classified windows.
|
|
1984
|
+ It produced a plain text file containing the sensor readings and timestamp
|
|
1985
|
+ for each of the 128 points in each window, as well as the manually-applied
|
|
1986
|
+ classification.
|
|
1987
|
+ It did not include information as to which windows were from the same sample,
|
|
1988
|
+ or the orginal user-supplied activity name.
|
|
1989
|
+\begin_inset Note Note
|
|
1990
|
+status open
|
|
1991
|
+
|
|
1992
|
+\begin_layout Plain Layout
|
|
1993
|
+Sample in appendix
|
|
1994
|
+\end_layout
|
|
1995
|
+
|
|
1996
|
+\end_inset
|
|
1997
|
+
|
|
1998
|
+
|
|
1999
|
+\end_layout
|
|
2000
|
+
|
|
2001
|
+\begin_layout Standard
|
|
2002
|
+A Java program was written which imported the exported data.
|
|
2003
|
+ A series of 'feature extractors' were written.
|
|
2004
|
+ These included the range, minimum, median, mean, maximum and absolute mean
|
|
2005
|
+\begin_inset Note Note
|
|
2006
|
+status open
|
|
2007
|
+
|
|
2008
|
+\begin_layout Plain Layout
|
|
2009
|
+explain
|
|
2010
|
+\end_layout
|
|
2011
|
+
|
|
2012
|
+\end_inset
|
|
2013
|
+
|
|
2014
|
+.
|
|
2015
|
+ Each of these was calculated over the set of 128 data points from each
|
|
2016
|
+ axis on each sensor, giving a total of 36 features.
|
|
2017
|
+ Before initial tests with these features were performed, it was planned
|
|
2018
|
+ to add further features including fast fourier transforms and energy, as
|
|
2019
|
+ suggested by multiple papers consulted in background research.
|
|
2020
|
+\end_layout
|
|
2021
|
+
|
|
2022
|
+\begin_layout Standard
|
|
2023
|
+The program was modified to export the features and classification of each
|
|
2024
|
+ window in ARFF
|
|
2025
|
+\begin_inset Note Note
|
|
2026
|
+status open
|
|
2027
|
+
|
|
2028
|
+\begin_layout Plain Layout
|
|
2029
|
+ref
|
|
2030
|
+\end_layout
|
|
2031
|
+
|
|
2032
|
+\end_inset
|
|
2033
|
+
|
|
2034
|
+ (Attribute Relation File Format).
|
|
2035
|
+ This is a format used by Weka
|
|
2036
|
+\begin_inset Note Note
|
|
2037
|
+status open
|
|
2038
|
+
|
|
2039
|
+\begin_layout Plain Layout
|
|
2040
|
+ref
|
|
2041
|
+\end_layout
|
|
2042
|
+
|
|
2043
|
+\end_inset
|
|
2044
|
+
|
|
2045
|
+, a popular data mining suite developed by the University of Waikato.
|
|
2046
|
+\begin_inset Note Note
|
|
2047
|
+status open
|
|
2048
|
+
|
|
2049
|
+\begin_layout Plain Layout
|
|
2050
|
+Sample in appendix
|
|
2051
|
+\end_layout
|
|
2052
|
+
|
|
2053
|
+\end_inset
|
|
2054
|
+
|
|
2055
|
+ The ARFF file was imported into Weka and analysed.
|
|
2056
|
+\end_layout
|
|
2057
|
+
|
|
2058
|
+\begin_layout Standard
|
|
2059
|
+It was found that Weka could correctly classified
|
|
2060
|
+\begin_inset Note Note
|
|
2061
|
+status open
|
|
2062
|
+
|
|
2063
|
+\begin_layout Plain Layout
|
|
2064
|
+99%
|
|
2065
|
+\end_layout
|
|
2066
|
+
|
|
2067
|
+\end_inset
|
|
2068
|
+
|
|
2069
|
+ of activities (measured using a
|
|
2070
|
+\begin_inset Note Note
|
|
2071
|
+status open
|
|
2072
|
+
|
|
2073
|
+\begin_layout Plain Layout
|
|
2074
|
+hold-one cross validation
|
|
2075
|
+\end_layout
|
|
2076
|
+
|
|
2077
|
+\end_inset
|
|
2078
|
+
|
|
2079
|
+) using a K-Nearest Network algorithm with K = 1.
|
|
2080
|
+ Furthermore, it became apparent that the error did not significantly increase
|
|
2081
|
+ when the 36 features were reduced to just four - the absolute mean and
|
|
2082
|
+ the range of the Y and Z accelerometer readings.
|
|
2083
|
+ Given the low cost of calculating these features and the very high detection
|
|
2084
|
+ rate, further research into other features such as FFTs was not performed.
|
|
2085
|
+\end_layout
|
|
2086
|
+
|
|
2087
|
+\begin_layout Standard
|
|
2088
|
+The application was again modified to read the exported data, calculate
|
|
2089
|
+ the four interesting features, and create a mapping of data points to activity.
|
|
2090
|
+ This map was then serialised and written to a file
|
|
2091
|
+\begin_inset Note Note
|
|
2092
|
+status open
|
|
2093
|
+
|
|
2094
|
+\begin_layout Plain Layout
|
|
2095
|
+expand
|
|
2096
|
+\end_layout
|
|
2097
|
+
|
|
2098
|
+\end_inset
|
|
2099
|
+
|
|
2100
|
+.
|
|
2101
|
+\end_layout
|
|
2102
|
+
|
|
2103
|
+\begin_layout Section
|
|
2104
|
+On-device classification
|
|
2105
|
+\end_layout
|
|
2106
|
+
|
|
2107
|
+\begin_layout Standard
|
|
2108
|
+The serialised model was bundled with a new version of the Sensor Logger
|
|
2109
|
+ application, along with a completely redesigned user interface.
|
|
2110
|
+ The application now consisted of a series of activities: on opening the
|
|
2111
|
+ application, the user is presented with an introduction activity which
|
|
2112
|
+ explains what the application does and the aims of the project; when they
|
|
2113
|
+ click the 'next' button a 10 second countdown is displayed with the instruction
|
|
2114
|
+ to put the phone away repeated.
|
|
2115
|
+ During the data collection period an animated pattern of dots is displayed
|
|
2116
|
+ so the user knows the application is still working if they look at it.
|
|
2117
|
+ When data collection is completed, the device adds a notification with
|
|
2118
|
+ sound and vibrate settings, and displays another progress screen while
|
|
2119
|
+ a background service analyses the data.
|
|
2120
|
+ The result of the classification is then displayed prominently and the
|
|
2121
|
+ user is presented with two buttons - one to confirm the classification
|
|
2122
|
+ and one to reject and correct it.
|
|
2123
|
+ The on-device classification (and any correction made to it) is submitted
|
|
2124
|
+ along with all the data previously submitted.
|
|
2125
|
+ The user is then presented with a 'Thank you' message, and a unique link
|
|
2126
|
+ to the project website where they can view graphs of their activities.
|
|
2127
|
+\begin_inset Note Note
|
|
2128
|
+status open
|
|
2129
|
+
|
|
2130
|
+\begin_layout Plain Layout
|
|
2131
|
+Mess!
|
|
2132
|
+\end_layout
|
|
2133
|
+
|
|
2134
|
+\end_inset
|
|
2135
|
+
|
|
2136
|
+
|
|
2137
|
+\end_layout
|
|
2138
|
+
|
|
2139
|
+\begin_layout Section
|
|
2140
|
+Activity Recorder application
|
|
2141
|
+\end_layout
|
|
2142
|
+
|
1760
|
2143
|
\begin_layout Part
|
1761
|
2144
|
Microphone, camera and Bluetooth
|
1762
|
2145
|
\end_layout
|
|
@@ -1766,7 +2149,7 @@ Places
|
1766
|
2149
|
\end_layout
|
1767
|
2150
|
|
1768
|
2151
|
\begin_layout Part
|
1769
|
|
-The Context Analyser
|
|
2152
|
+The Context Analyser applications
|
1770
|
2153
|
\end_layout
|
1771
|
2154
|
|
1772
|
2155
|
\begin_layout Standard
|
|
@@ -1936,13 +2319,13 @@ Finally, the applications should be evaluated based on their resource usage.
|
1936
|
2319
|
|
1937
|
2320
|
\begin_layout Part
|
1938
|
2321
|
Conclusion
|
|
2322
|
+\end_layout
|
|
2323
|
+
|
|
2324
|
+\begin_layout Standard
|
1939
|
2325
|
\begin_inset Newpage pagebreak
|
1940
|
2326
|
\end_inset
|
1941
|
2327
|
|
1942
|
2328
|
|
1943
|
|
-\end_layout
|
1944
|
|
-
|
1945
|
|
-\begin_layout Standard
|
1946
|
2329
|
\begin_inset CommandInset bibtex
|
1947
|
2330
|
LatexCommand bibtex
|
1948
|
2331
|
bibfiles "/home/chris/Uni/project/papers/project"
|