My Notes

Mostly for personal use. These notes are not clearly written.

Display Output Online

Activity recorder is brilliant but it needs a way to display output that is compelling enough to get people more interested. The places application is not ideal and it doesn’t need to be on the phone. Google Lattitude’s history function has really got that right. Look at past data on a cool map, provide a summary analysis and make the data available to other applications. The ideal would be update Lattitude locations with entries containing extra state parameters. They have an API which lets you update locations but you can’t add extra parameters , I wonder if it is an undocumented feature. A second best is to post icons to Google MyMaps , still a good and simple option with good data presentation. Google MyMaps does let you add extended parameters like the Latitude data I have extracted and uploaded.

States To Report

Don’t report states that don’t exist for at least 3 minutes or so. Perhaps 3 x 56.5 sec sampling interval. Do different levels of sampling and analysis depending on state. Everyone reports states but nobody provides any scalar information on states. Seems useful if scalar information can be captured as well.

• Charging - 100% accurate measure. Not measuring person but could be charging in vehicle so could check for vehicle movement. Then it would be two states charge or travel.
• Uncarried - (Is there a better word?) should be very accurate. Standard deviation on every axis is tiny also orientation is same as last sample. To increase battery life, once detected, next sampling interval only sample accelerometers once then check if orientation is the same. Report orientation as well which may be useful to infer other things in post analysis e.g. if its on its back that is normal but if it is some other angle it is in a handbag.
• Walking - All papers suggest this is the easiest movement to detect. Accelerations are large in the vertical axis when walking and larger than any other activity. Rather than look for max - min look for large standard deviation. Much more useful if you count steps as well. Horizontal direction vector from compass would be good. Standard deviation on direction vector angle would be good too. If the SD is low then going in same direction for sampling interval. No one in publications does this. Will need increased processing while walking to do this. So report “walking 10 minutes 500 (or whatever) steps heading headingSD (i.e SD) 10 degrees.. This will then include running, jogging etc. from rate. Most people classify running separately.
• Paddling - Smooth accelerations in horizontal axis without large change in direction.
• Travelling. Maybe not necessary. GPS does this well. It would be nice to know vehicle vs bicycle. Because Car/Bus can change a lot in speed over 6.5 secs an average acceleration that exceeded gravity must mean travelling in a vehicle as an acceleration that exists for a long time means a big change in velocity. Nobody uses this.
• Active. Something is happening but it hasn’t been classified. Change of orientation during sampling interval which is detected by average acceleration over 6.5 secs being less than gravity should mean something. Nobody uses this. Could include vigorousness measure which is SD of acceleration vector and directionality vertical and directionality horizontal which is SD of rotation. Phone is attached to body so vigour measure is roughly proportional to energy consumed in moving body which is important for exercise levels. Did I spend the day doing physical activity or was I sedentary? Data reported is Active- Level 0.2G Vertical Directionality 20 degrees Horizontal Directionality 90 degrees.
• Standing vs Sitting (Not) Seems useful but can’t be detected with phone in breast pock according to Parkka2006. That’s why Activity recorder gets it wrong so much. I often have the phone in breast pocket and training data was acquired with phone in hip pocket.

This small set should make it possible to get a high degree of classification accuracy that corresponds to user perception of accuracy and is enough to be useful. Perception of accuracy is different to measures researchers are currently using (see Consolvo paper).

Accelerometer Sampling

Assumes accelerometer can be sampled accurately at 20Hz. 20 Hz is slower than other people use. Need to check if 20Hz gets significant signal distortion from folding about Nyquist frequency. Signals look good on “Sensor Logger” plots as steps can be clearly seen and it looks sensible. Slower is better from power point of view and slower will increase sampling interval reliability. Minimum sampling interval can be inferred from some of the publication below that do FFT analysis.

There is no way to specify accurate sample intervals. This guy gets 8 milliseconds to 200 milliseconds (200 milliseconds may be during garbage collection). The Stroke Coach developer argues Android is not a suitable platform for these and other reasons but two rowing applications exist. These are And/Oar and Robostroke . Some people are keen . Robostroke states “Talos Rowing performance and precision may depend on the strength and specs of the Android device it runs on.” suggesting they are having trouble with measuring accelerometers. This guy uses event.timestamp which means you can get it timed accurately which is probably enough. Delete samples that have sample intervals above a threshold and do interpolation to even out the rest if time based analysis e.g. FFT.

Accelerometers ought to be auto-calibrated say once per day. An accurate gravity value is useful and gravity changes from place to place. When Charging is detected get a longish sample say 10 minutes and get average on each channel, make sure each new sample is not much different to current mean so we know phone wasn’t moved then combine to measure gravity.

Fast Fourier Transform

Fast fourier transform is ideal for walking and should reveal pace but it is computationally intensive and therefore power hungry. Would be slow in Java. Someone has done an FFT implementation in a Android C application. It looks useful .

Paper Review Notes

Everyone uses classifiers, mostly they don’t try to understand the underlying physics but just look for patterns. Each axis of accelerometer is considered separately which works for many because device attachment to body is always the same but not for a phone. They all claim great accuracy but it is almost certainly using criteria that don’t match practical use or people’s perceptions of accuracy. Consolvo 2008 discusses incorrect criteria for accuracy.

• Bao2004 Classification lots of interesting ideas. Very computationally intensive and many accelerometers.
• Caros2005 Propose simple algorithm that is tested to work reasonably well. Uses single axis accelerometer.
• Chouidhury 2008 Complicated hardware and sophisticated processing. Used classifiers. Found accelerometer data and microphone data to be most useful.
• Consolvo 2008 Discusses types of errors that produce user dissatisfaction and that this is not how error rates are normally measured. Mentions importance of allowing users to edit inferred information.
• Garakani2009 This looks to be the system Chris based his on. Windows. Uses FFT without crushing battery, samples continuously at 25Hz and can last 10 hours.
• Hein2008 Supervised clustering. Seems to use each axis of accelerometer separately.
• Huynh2005 Good review of accelerometer parameters used to detect activities in Introduction. Looks at ideal sampling intervals. Found that FFT works best. Gives information on best frequencies to use. Considered each axis separately as does everyone else above.
• Lester2005 Uses temporal data in form of hidden Markov models to increase accuracy. Also uses each axis of accelerometer independently. Checks lots of sensors accelerometers are good, audio is good compass useless but if compass was combined with accelerometer to get direction I bet it is different.
• Lester2006 reckons shoulder, hip and wrist are all pretty similar for recognition. Mentioned that most of the time data was recorded there was no identifiable activity.
• Mathie2004 Hierarchical classifier. Activities tightly defined real activities much loser e.g. standing involves occasioal steps.
• Parkka2006 Accelerometer on rucksack shoulder strap saw same signal for sitting and standing. Also classifies rowing quite well.
• Ravi2005 Liked plurality voting.
• Schmidt2008 Describes Iphone system that achieves great accuracy sampling 200Hz and doing FFT.
• Song2005 uses wavelets.
• Wang2009 Discusses problem of people putting their phone in different positions. Uses only SD of accelerations for activities still, vehicle, walking, running. Works OK. Explains how to recognise speech from other sounds.